2023-06-09T04:23:30.729 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd 2023-06-09T04:23:30.730 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} --owner scheduled_xiubli@teuthology --archive /home/teuthworker/archive/xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7299733 --verbose --name xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi -- /home/teuthworker/archive/xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7299733/orig.config.yaml 2023-06-09T04:23:30.782 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-09T04:23:30.873 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7299733 branch: wip-lxb-fscrypt-20230607-0901 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: null first_in_suite: false job_id: '7299733' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-lxb-fscrypt-20230607-0901 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lxb-fscrypt-20230607-0901 sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b owner: scheduled_xiubli@teuthology priority: 110 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-lxb-fscrypt-20230607-0901 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61 timestamp: 2023-06-09_04:16:27 tube: smithi user: xiubli verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-09T04:23:30.873 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa; will attempt to use it 2023-06-09T04:23:30.874 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa/tasks 2023-06-09T04:23:30.875 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-09T04:23:30.876 INFO:teuthology.task.internal:Checking packages... 2023-06-09T04:23:30.903 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a02638f808b8fe11bccdd01222c0e6e2eb71120b' 2023-06-09T04:23:30.904 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-09T04:23:30.904 INFO:teuthology.packaging:ref: None 2023-06-09T04:23:30.904 INFO:teuthology.packaging:tag: None 2023-06-09T04:23:30.905 INFO:teuthology.packaging:branch: wip-lxb-fscrypt-20230607-0901 2023-06-09T04:23:30.905 INFO:teuthology.packaging:sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-09T04:23:30.905 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-lxb-fscrypt-20230607-0901 2023-06-09T04:23:31.177 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4313.ga02638f8 2023-06-09T04:23:31.177 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-09T04:23:31.188 INFO:teuthology.task.internal:no buildpackages task found 2023-06-09T04:23:31.188 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-09T04:23:31.219 INFO:teuthology.task.internal:Saving configuration 2023-06-09T04:23:31.235 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-09T04:23:31.249 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-09T04:23:31.276 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7299733', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-09 04:16:32.616704', 'locked_by': 'scheduled_xiubli@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-09T04:23:31.295 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7299733', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-09 04:16:32.614648', 'locked_by': 'scheduled_xiubli@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-09T04:23:31.296 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-09T04:23:31.306 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-09T04:23:31.306 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-09T04:23:31.306 INFO:teuthology.run_tasks:Running task console_log... 2023-06-09T04:23:31.387 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8549436f70>, signals=[15]) 2023-06-09T04:23:31.391 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-09T04:23:31.401 INFO:teuthology.task.internal:Opening connections... 2023-06-09T04:23:31.401 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:23:31.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T04:23:31.471 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:23:31.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T04:23:31.536 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-09T04:23:31.548 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-06-09T04:23:31.568 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-06-09T04:23:31.569 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-06-09T04:23:31.628 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Red Hat Enterprise Linux" 2023-06-09T04:23:31.628 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8.6 (Ootpa)" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:ID="rhel" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="fedora" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8.6" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-09T04:23:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.redhat.com/" 2023-06-09T04:23:31.630 INFO:teuthology.orchestra.run.smithi092.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-09T04:23:31.630 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-09T04:23:31.630 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:23:31.630 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-09T04:23:31.630 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-09T04:23:31.630 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-09T04:23:31.630 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-09T04:23:31.631 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-06-09T04:23:31.680 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-06-09T04:23:31.698 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-06-09T04:23:31.698 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-06-09T04:23:31.753 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2023-06-09T04:23:31.754 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.6 (Ootpa)" 2023-06-09T04:23:31.754 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2023-06-09T04:23:31.754 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2023-06-09T04:23:31.754 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.6" 2023-06-09T04:23:31.754 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-06-09T04:23:31.754 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-09T04:23:31.755 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-06-09T04:23:31.755 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-09T04:23:31.755 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2023-06-09T04:23:31.755 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-09T04:23:31.755 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-09T04:23:31.755 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:23:31.755 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-09T04:23:31.756 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-09T04:23:31.756 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-09T04:23:31.756 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-09T04:23:31.756 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-06-09T04:23:31.780 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-09T04:23:31.796 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-09T04:23:31.808 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-09T04:23:31.808 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2023-06-09T04:23:31.811 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2023-06-09T04:23:31.827 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-09T04:23:31.837 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-09T04:23:31.838 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2023-06-09T04:23:31.870 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2023-06-09T04:23:31.888 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-09T04:23:31.951 INFO:teuthology.run_tasks:Running task kernel... 2023-06-09T04:23:31.973 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-06-09T04:23:31.974 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-06-09T04:23:31.974 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-06-09T04:23:31.975 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-09T04:23:31.976 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-09T04:23:31.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:23:31.993 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2023-06-09T04:23:31.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:23:31.994 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-06-09T04:23:32.048 INFO:teuthology.orchestra.run.smithi123.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:32.049 INFO:teuthology.task.kernel:Running kernel on smithi123: 4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:32.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2023-06-09T04:23:32.053 INFO:teuthology.orchestra.run.smithi092.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:32.054 INFO:teuthology.task.kernel:Running kernel on smithi092: 4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:32.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2023-06-09T04:23:33.222 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-06-09T04:23:33.222 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-06-09T04:23:33.291 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-06-09T04:23:33.292 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2023-06-09T04:23:36.842 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-09T04:23:36.967 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-09T04:23:38.584 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 16 MB 00:01 2023-06-09T04:23:39.409 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 16 MB 00:02 2023-06-09T04:23:41.984 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 542 kB/s | 24 kB 00:00 2023-06-09T04:23:42.619 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-09T04:23:42.673 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-06-09T04:23:42.674 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2023-06-09T04:23:42.674 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-06-09T04:23:42.782 DEBUG:teuthology.orchestra.run.smithi123:> echo no | sudo yum reinstall kernel || true 2023-06-09T04:23:42.892 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 197 kB/s | 24 kB 00:00 2023-06-09T04:23:43.116 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-06-09T04:23:43.116 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2023-06-09T04:23:43.380 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:01 ago on Fri 09 Jun 2023 04:23:42 AM UTC. 2023-06-09T04:23:43.421 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-09T04:23:43.454 INFO:teuthology.orchestra.run.smithi123.stderr:Error: No packages marked for reinstall. 2023-06-09T04:23:43.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum reinstall -y kernel || true 2023-06-09T04:23:43.536 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-09T04:23:43.595 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-06-09T04:23:43.596 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2023-06-09T04:23:43.596 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-06-09T04:23:43.703 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2023-06-09T04:23:43.829 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-06-09T04:23:43.829 INFO:teuthology.orchestra.run.smithi123.stdout:Unable to read consumer identity 2023-06-09T04:23:44.033 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-06-09T04:23:44.033 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-06-09T04:23:44.078 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:02 ago on Fri 09 Jun 2023 04:23:42 AM UTC. 2023-06-09T04:23:44.118 INFO:teuthology.orchestra.run.smithi123.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-09T04:23:44.153 INFO:teuthology.orchestra.run.smithi123.stderr:Error: No packages marked for reinstall. 2023-06-09T04:23:44.198 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2023-06-09T04:23:44.278 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:44.278 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:44.278 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-09T04:23:44.278 DEBUG:teuthology.parallel:result is None 2023-06-09T04:23:44.288 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Fri 09 Jun 2023 04:23:42 AM UTC. 2023-06-09T04:23:44.332 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-09T04:23:44.369 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-06-09T04:23:44.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2023-06-09T04:23:44.743 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-06-09T04:23:44.743 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-06-09T04:23:44.991 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Fri 09 Jun 2023 04:23:42 AM UTC. 2023-06-09T04:23:45.032 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-09T04:23:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-06-09T04:23:45.108 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2023-06-09T04:23:45.188 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:45.189 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-09T04:23:45.189 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-09T04:23:45.189 DEBUG:teuthology.parallel:result is None 2023-06-09T04:23:45.189 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-09T04:23:45.196 INFO:teuthology.task.internal:Creating test directory... 2023-06-09T04:23:45.196 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-09T04:23:45.198 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-09T04:23:45.217 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-09T04:23:45.225 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-09T04:23:45.242 INFO:teuthology.task.internal:Creating archive directory... 2023-06-09T04:23:45.242 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-09T04:23:45.255 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-09T04:23:45.280 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-09T04:23:45.300 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-09T04:23:45.301 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-09T04:23:45.313 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-09T04:23:45.341 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T04:23:45.352 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T04:23:45.384 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T04:23:45.401 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T04:23:45.403 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-09T04:23:45.412 INFO:teuthology.task.internal:Configuring sudo... 2023-06-09T04:23:45.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-09T04:23:45.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-09T04:23:45.442 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-09T04:23:45.453 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-09T04:23:45.453 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-09T04:23:45.481 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-09T04:23:45.498 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T04:23:45.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T04:23:45.630 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T04:23:45.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T04:23:45.754 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:23:45.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-09T04:23:45.820 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T04:23:45.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T04:23:45.920 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T04:23:46.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T04:23:46.046 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:23:46.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-09T04:23:46.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2023-06-09T04:23:46.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2023-06-09T04:23:46.172 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-09T04:23:46.219 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-09T04:23:46.662 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-09T04:23:46.672 INFO:teuthology.task.internal:Starting timer... 2023-06-09T04:23:46.673 INFO:teuthology.run_tasks:Running task pcp... 2023-06-09T04:23:46.690 INFO:teuthology.run_tasks:Running task selinux... 2023-06-09T04:23:46.703 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-06-09T04:23:46.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2023-06-09T04:23:46.768 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2023-06-09T04:23:46.770 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2023-06-09T04:23:46.821 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2023-06-09T04:23:46.822 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-09T04:23:46.823 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2023-06-09T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2023-06-09T04:23:46.848 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2023-06-09T04:23:46.874 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2023-06-09T04:23:46.875 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2023-06-09T04:23:46.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-09T04:23:46.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:23:46.916 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-09T04:23:46.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:23:46.944 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-09T04:23:46.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2023-06-09T04:23:46.983 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2023-06-09T04:23:47.010 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-09T04:23:47.020 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-06-09T04:23:47.025 INFO:teuthology.repo_utils:Fetching main from origin 2023-06-09T04:23:47.146 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-09T04:23:47.165 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-09T04:23:47.167 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-09T04:31:59.467 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2023-06-09T04:31:59.468 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-06-09T04:31:59.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T04:31:59.537 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-06-09T04:31:59.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-06-09T04:31:59.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T04:31:59.672 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-06-09T04:31:59.697 INFO:teuthology.run_tasks:Running task clock... 2023-06-09T04:31:59.708 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-09T04:31:59.709 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-09T04:31:59.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-09T04:31:59.711 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-09T04:31:59.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-09T04:31:59.739 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-09T04:31:59.754 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-09T04:31:59.768 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-09T04:31:59.783 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-09T04:31:59.821 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2023-06-09T04:31:59.833 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2023-06-09T04:31:59.847 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-09T04:31:59.853 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2023-06-09T04:31:59.861 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-09T04:31:59.865 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2023-06-09T04:31:59.879 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-09T04:31:59.892 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-09T04:31:59.944 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-06-09T04:31:59.948 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-09T04:31:59.948 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-06-09T04:31:59.948 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.948 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.949 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.955 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-06-09T04:31:59.959 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-09T04:31:59.960 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-06-09T04:31:59.960 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.960 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.960 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.960 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T04:31:59.961 INFO:teuthology.run_tasks:Running task install... 2023-06-09T04:31:59.969 DEBUG:teuthology.task.install:project ceph 2023-06-09T04:31:59.970 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}} 2023-06-09T04:31:59.970 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'} 2023-06-09T04:31:59.970 INFO:teuthology.task.install:Using flavor: default 2023-06-09T04:31:59.976 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-09T04:31:59.976 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-09T04:31:59.977 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'tag': None, 'wait_for_package': False} 2023-06-09T04:31:59.977 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-09T04:31:59.979 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'tag': None, 'wait_for_package': False} 2023-06-09T04:31:59.980 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-09T04:32:00.203 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/ 2023-06-09T04:32:00.203 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4313.ga02638f8 2023-06-09T04:32:00.207 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/ 2023-06-09T04:32:00.207 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4313.ga02638f8 2023-06-09T04:32:00.377 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-09T04:32:00.377 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:32:00.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-09T04:32:00.380 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-09T04:32:00.380 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:32:00.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-09T04:32:00.407 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-09T04:32:00.408 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a02638f808b8fe11bccdd01222c0e6e2eb71120b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-09T04:32:00.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-09T04:32:00.409 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a02638f808b8fe11bccdd01222c0e6e2eb71120b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-09T04:32:00.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-09T04:32:00.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-09T04:32:00.552 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-09T04:32:00.553 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-09T04:32:00.617 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2023-06-09T04:32:00.618 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2023-06-09T04:32:00.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2023-06-09T04:32:00.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2023-06-09T04:32:00.989 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-06-09T04:32:00.989 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-06-09T04:32:01.252 INFO:teuthology.orchestra.run.smithi092.stdout:44 files removed 2023-06-09T04:32:01.259 INFO:teuthology.orchestra.run.smithi123.stdout:44 files removed 2023-06-09T04:32:01.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-09T04:32:01.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-09T04:32:01.627 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-06-09T04:32:01.633 INFO:teuthology.orchestra.run.smithi123.stdout:Updating Subscription Management repositories. 2023-06-09T04:32:02.378 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 213 kB/s | 80 kB 00:00 2023-06-09T04:32:02.382 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 215 kB/s | 80 kB 00:00 2023-06-09T04:32:02.704 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-06-09T04:32:02.750 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-06-09T04:32:03.047 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 4.6 kB/s | 1.5 kB 00:00 2023-06-09T04:32:03.076 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 4.9 kB/s | 1.5 kB 00:00 2023-06-09T04:32:05.422 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-09T04:32:05.609 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-09T04:32:07.189 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 16 MB 00:01 2023-06-09T04:32:10.214 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 16 MB 00:04 2023-06-09T04:32:10.605 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 374 kB/s | 24 kB 00:00 2023-06-09T04:32:11.485 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-06-09T04:32:13.768 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 423 kB/s | 24 kB 00:00 2023-06-09T04:32:14.686 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-06-09T04:32:19.485 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-06-09T04:32:22.845 INFO:teuthology.orchestra.run.smithi123.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-06-09T04:32:30.523 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T04:32:30.524 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T04:32:30.538 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-09T04:32:30.539 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-09T04:32:30.540 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-09T04:32:30.542 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-09T04:32:30.544 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-09T04:32:30.547 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-09T04:32:30.548 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-09T04:32:30.679 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-06-09T04:32:30.691 INFO:teuthology.orchestra.run.smithi092.stdout:=========================================================================================================================== 2023-06-09T04:32:30.691 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-06-09T04:32:30.691 INFO:teuthology.orchestra.run.smithi092.stdout:=========================================================================================================================== 2023-06-09T04:32:30.692 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-06-09T04:32:30.692 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-09T04:32:30.692 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-09T04:32:30.692 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.4 k 2023-06-09T04:32:30.692 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 5.4 M 2023-06-09T04:32:30.692 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 848 k 2023-06-09T04:32:30.692 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 166 k 2023-06-09T04:32:30.693 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 1.5 M 2023-06-09T04:32:30.693 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 128 k 2023-06-09T04:32:30.693 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 1.7 M 2023-06-09T04:32:30.693 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 7.4 M 2023-06-09T04:32:30.693 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 47 k 2023-06-09T04:32:30.693 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 13 M 2023-06-09T04:32:30.693 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 41 M 2023-06-09T04:32:30.694 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 258 k 2023-06-09T04:32:30.694 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 160 k 2023-06-09T04:32:30.694 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 258 k 2023-06-09T04:32:30.694 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 18 k 2023-06-09T04:32:30.694 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-09T04:32:30.694 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-09T04:32:30.694 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-09T04:32:30.695 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 31 k 2023-06-09T04:32:30.695 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 771 k 2023-06-09T04:32:30.695 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 139 k 2023-06-09T04:32:30.695 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-09T04:32:30.695 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-09T04:32:30.695 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-09T04:32:30.696 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-09T04:32:30.696 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-09T04:32:30.696 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-09T04:32:30.696 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 197 k 2023-06-09T04:32:30.696 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 370 k 2023-06-09T04:32:30.697 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 365 k 2023-06-09T04:32:30.697 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 107 k 2023-06-09T04:32:30.697 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 94 k 2023-06-09T04:32:30.697 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.4 M 2023-06-09T04:32:30.697 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 179 k 2023-06-09T04:32:30.697 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-09T04:32:30.697 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.7 M 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.6 M 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 22 M 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 23 k 2023-06-09T04:32:30.698 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 2.3 M 2023-06-09T04:32:30.699 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 240 k 2023-06-09T04:32:30.699 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 4.6 M 2023-06-09T04:32:30.699 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 18 M 2023-06-09T04:32:30.699 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 15 k 2023-06-09T04:32:30.699 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 24 k 2023-06-09T04:32:30.699 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-09T04:32:30.699 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-09T04:32:30.700 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-09T04:32:30.700 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-09T04:32:30.700 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-09T04:32:30.700 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 137 k 2023-06-09T04:32:30.700 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-09T04:32:30.700 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-09T04:32:30.700 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-09T04:32:30.701 INFO:teuthology.orchestra.run.smithi092.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-09T04:32:30.701 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-09T04:32:30.701 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-09T04:32:30.701 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 473 k 2023-06-09T04:32:30.701 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-09T04:32:30.701 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.5 M 2023-06-09T04:32:30.701 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-09T04:32:30.702 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-09T04:32:30.702 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-09T04:32:30.702 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-09T04:32:30.702 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-09T04:32:30.702 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-09T04:32:30.702 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-09T04:32:30.702 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-09T04:32:30.703 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-09T04:32:30.703 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-09T04:32:30.703 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-09T04:32:30.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-09T04:32:30.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-09T04:32:30.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-09T04:32:30.703 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-09T04:32:30.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-09T04:32:30.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 45 k 2023-06-09T04:32:30.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 119 k 2023-06-09T04:32:30.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-09T04:32:30.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-09T04:32:30.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-09T04:32:30.704 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-09T04:32:30.705 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T04:32:30.706 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-09T04:32:30.706 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-09T04:32:30.706 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-09T04:32:30.706 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-09T04:32:30.706 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-09T04:32:30.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-09T04:32:30.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-09T04:32:30.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-09T04:32:30.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-09T04:32:30.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-09T04:32:30.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-09T04:32:30.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T04:32:30.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-09T04:32:30.709 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-09T04:32:30.709 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-09T04:32:30.709 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-09T04:32:30.709 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-09T04:32:30.709 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-09T04:32:30.709 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-09T04:32:30.709 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-09T04:32:30.710 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-09T04:32:30.711 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-09T04:32:30.711 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-09T04:32:30.711 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling module streams: 2023-06-09T04:32:30.711 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql 10 2023-06-09T04:32:30.711 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:32:30.711 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-06-09T04:32:30.711 INFO:teuthology.orchestra.run.smithi092.stdout:=========================================================================================================================== 2023-06-09T04:32:30.712 INFO:teuthology.orchestra.run.smithi092.stdout:Install 128 Packages 2023-06-09T04:32:30.712 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 3 Packages 2023-06-09T04:32:30.712 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:32:30.712 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 197 M 2023-06-09T04:32:30.713 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-06-09T04:32:31.159 INFO:teuthology.orchestra.run.smithi092.stdout:(1/131): ceph-18.0.0-4313.ga02638f8.el8.x86_64. 48 kB/s | 6.4 kB 00:00 2023-06-09T04:32:31.342 INFO:teuthology.orchestra.run.smithi092.stdout:(2/131): ceph-fuse-18.0.0-4313.ga02638f8.el8.x8 4.5 MB/s | 848 kB 00:00 2023-06-09T04:32:31.401 INFO:teuthology.orchestra.run.smithi092.stdout:(3/131): ceph-immutable-object-cache-18.0.0-431 2.8 MB/s | 166 kB 00:00 2023-06-09T04:32:31.635 INFO:teuthology.orchestra.run.smithi092.stdout:(4/131): ceph-mds-18.0.0-4313.ga02638f8.el8.x86 9.8 MB/s | 2.3 MB 00:00 2023-06-09T04:32:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:(5/131): ceph-mgr-18.0.0-4313.ga02638f8.el8.x86 7.4 MB/s | 1.5 MB 00:00 2023-06-09T04:32:32.352 INFO:teuthology.orchestra.run.smithi092.stdout:(6/131): ceph-mon-18.0.0-4313.ga02638f8.el8.x86 9.1 MB/s | 4.6 MB 00:00 2023-06-09T04:32:32.552 INFO:teuthology.orchestra.run.smithi092.stdout:(7/131): ceph-common-18.0.0-4313.ga02638f8.el8. 14 MB/s | 22 MB 00:01 2023-06-09T04:32:33.352 INFO:teuthology.orchestra.run.smithi092.stdout:(8/131): ceph-radosgw-18.0.0-4313.ga02638f8.el8 16 MB/s | 13 MB 00:00 2023-06-09T04:32:33.394 INFO:teuthology.orchestra.run.smithi092.stdout:(9/131): ceph-selinux-18.0.0-4313.ga02638f8.el8 588 kB/s | 24 kB 00:00 2023-06-09T04:32:33.709 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T04:32:33.710 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T04:32:33.724 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-09T04:32:33.725 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-09T04:32:33.726 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-09T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-09T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-09T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-09T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-09T04:32:33.862 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-06-09T04:32:33.874 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-06-09T04:32:33.875 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-06-09T04:32:33.875 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-06-09T04:32:33.875 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2023-06-09T04:32:33.875 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-09T04:32:33.875 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-09T04:32:33.875 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.4 k 2023-06-09T04:32:33.875 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 5.4 M 2023-06-09T04:32:33.876 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 848 k 2023-06-09T04:32:33.876 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 166 k 2023-06-09T04:32:33.876 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 1.5 M 2023-06-09T04:32:33.876 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 128 k 2023-06-09T04:32:33.876 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 1.7 M 2023-06-09T04:32:33.876 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 7.4 M 2023-06-09T04:32:33.877 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 47 k 2023-06-09T04:32:33.877 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 13 M 2023-06-09T04:32:33.877 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 41 M 2023-06-09T04:32:33.877 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 258 k 2023-06-09T04:32:33.877 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 160 k 2023-06-09T04:32:33.877 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 258 k 2023-06-09T04:32:33.877 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 18 k 2023-06-09T04:32:33.878 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-09T04:32:33.878 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-09T04:32:33.878 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-09T04:32:33.878 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 31 k 2023-06-09T04:32:33.878 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 771 k 2023-06-09T04:32:33.878 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 139 k 2023-06-09T04:32:33.878 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-09T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-09T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-09T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-09T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-09T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-09T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 197 k 2023-06-09T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 370 k 2023-06-09T04:32:33.880 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 365 k 2023-06-09T04:32:33.880 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 107 k 2023-06-09T04:32:33.880 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 94 k 2023-06-09T04:32:33.880 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.4 M 2023-06-09T04:32:33.880 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 179 k 2023-06-09T04:32:33.880 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-09T04:32:33.880 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2023-06-09T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-09T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.7 M 2023-06-09T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.6 M 2023-06-09T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-06-09T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-09T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 22 M 2023-06-09T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 23 k 2023-06-09T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 2.3 M 2023-06-09T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 240 k 2023-06-09T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 4.6 M 2023-06-09T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 18 M 2023-06-09T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 15 k 2023-06-09T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 24 k 2023-06-09T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-09T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-09T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-09T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-09T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-09T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 137 k 2023-06-09T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-09T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-09T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-09T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-09T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-09T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-09T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 473 k 2023-06-09T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-09T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.5 M 2023-06-09T04:32:33.885 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-09T04:32:33.885 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-09T04:32:33.885 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-09T04:32:33.885 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-09T04:32:33.885 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-09T04:32:33.885 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-09T04:32:33.885 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-09T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-09T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-09T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-09T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-09T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-09T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-09T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-09T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-09T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-09T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 45 k 2023-06-09T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 119 k 2023-06-09T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-09T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-09T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-09T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-09T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-09T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-09T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-09T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-09T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-09T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-09T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-09T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-09T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-09T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-09T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-09T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-09T04:32:33.890 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-09T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-09T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-09T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-09T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-09T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-09T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-09T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-09T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-09T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-09T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-09T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-09T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-09T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-09T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-09T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-09T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-09T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-09T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-09T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-09T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-09T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-09T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-09T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-09T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stdout:Enabling module streams: 2023-06-09T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql 10 2023-06-09T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-06-09T04:32:33.895 INFO:teuthology.orchestra.run.smithi123.stdout:=========================================================================================================================== 2023-06-09T04:32:33.895 INFO:teuthology.orchestra.run.smithi123.stdout:Install 128 Packages 2023-06-09T04:32:33.895 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 3 Packages 2023-06-09T04:32:33.895 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:32:33.896 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 197 M 2023-06-09T04:32:33.896 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-06-09T04:32:34.312 INFO:teuthology.orchestra.run.smithi123.stdout:(1/131): ceph-18.0.0-4313.ga02638f8.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-06-09T04:32:34.553 INFO:teuthology.orchestra.run.smithi092.stdout:(10/131): ceph-osd-18.0.0-4313.ga02638f8.el8.x8 8.1 MB/s | 18 MB 00:02 2023-06-09T04:32:34.603 INFO:teuthology.orchestra.run.smithi092.stdout:(11/131): cephfs-mirror-18.0.0-4313.ga02638f8.e 5.1 MB/s | 258 kB 00:00 2023-06-09T04:32:34.703 INFO:teuthology.orchestra.run.smithi092.stdout:(12/131): libcephfs-devel-18.0.0-4313.ga02638f8 310 kB/s | 31 kB 00:00 2023-06-09T04:32:35.028 INFO:teuthology.orchestra.run.smithi092.stdout:(13/131): libcephfs2-18.0.0-4313.ga02638f8.el8. 2.3 MB/s | 771 kB 00:00 2023-06-09T04:32:35.112 INFO:teuthology.orchestra.run.smithi092.stdout:(14/131): libcephsqlite-18.0.0-4313.ga02638f8.e 1.6 MB/s | 137 kB 00:00 2023-06-09T04:32:35.188 INFO:teuthology.orchestra.run.smithi092.stdout:(15/131): librados-devel-18.0.0-4313.ga02638f8. 1.8 MB/s | 139 kB 00:00 2023-06-09T04:32:35.329 INFO:teuthology.orchestra.run.smithi092.stdout:(16/131): libradosstriper1-18.0.0-4313.ga02638f 3.3 MB/s | 473 kB 00:00 2023-06-09T04:32:35.337 INFO:teuthology.orchestra.run.smithi123.stdout:(2/131): ceph-fuse-18.0.0-4313.ga02638f8.el8.x8 827 kB/s | 848 kB 00:01 2023-06-09T04:32:35.513 INFO:teuthology.orchestra.run.smithi123.stdout:(3/131): ceph-base-18.0.0-4313.ga02638f8.el8.x8 3.4 MB/s | 5.4 MB 00:01 2023-06-09T04:32:35.563 INFO:teuthology.orchestra.run.smithi123.stdout:(4/131): ceph-immutable-object-cache-18.0.0-431 736 kB/s | 166 kB 00:00 2023-06-09T04:32:35.697 INFO:teuthology.orchestra.run.smithi092.stdout:(17/131): ceph-base-18.0.0-4313.ga02638f8.el8.x 1.2 MB/s | 5.4 MB 00:04 2023-06-09T04:32:35.814 INFO:teuthology.orchestra.run.smithi092.stdout:(18/131): python3-ceph-argparse-18.0.0-4313.ga0 384 kB/s | 45 kB 00:00 2023-06-09T04:32:35.980 INFO:teuthology.orchestra.run.smithi123.stdout:(5/131): ceph-mds-18.0.0-4313.ga02638f8.el8.x86 4.9 MB/s | 2.3 MB 00:00 2023-06-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi092.stdout:(19/131): python3-ceph-common-18.0.0-4313.ga026 593 kB/s | 119 kB 00:00 2023-06-09T04:32:36.523 INFO:teuthology.orchestra.run.smithi092.stdout:(20/131): ceph-test-18.0.0-4313.ga02638f8.el8.x 13 MB/s | 41 MB 00:03 2023-06-09T04:32:36.548 INFO:teuthology.orchestra.run.smithi092.stdout:(21/131): python3-cephfs-18.0.0-4313.ga02638f8. 370 kB/s | 197 kB 00:00 2023-06-09T04:32:36.563 INFO:teuthology.orchestra.run.smithi123.stdout:(6/131): ceph-mon-18.0.0-4313.ga02638f8.el8.x86 7.9 MB/s | 4.6 MB 00:00 2023-06-09T04:32:36.631 INFO:teuthology.orchestra.run.smithi092.stdout:(22/131): librgw2-18.0.0-4313.ga02638f8.el8.x86 5.0 MB/s | 6.5 MB 00:01 2023-06-09T04:32:36.657 INFO:teuthology.orchestra.run.smithi092.stdout:(23/131): python3-rados-18.0.0-4313.ga02638f8.e 2.7 MB/s | 370 kB 00:00 2023-06-09T04:32:36.680 INFO:teuthology.orchestra.run.smithi123.stdout:(7/131): ceph-mgr-18.0.0-4313.ga02638f8.el8.x86 1.4 MB/s | 1.5 MB 00:01 2023-06-09T04:32:36.682 INFO:teuthology.orchestra.run.smithi092.stdout:(24/131): python3-rgw-18.0.0-4313.ga02638f8.el8 2.1 MB/s | 107 kB 00:00 2023-06-09T04:32:36.707 INFO:teuthology.orchestra.run.smithi092.stdout:(25/131): rbd-fuse-18.0.0-4313.ga02638f8.el8.x8 1.8 MB/s | 94 kB 00:00 2023-06-09T04:32:36.774 INFO:teuthology.orchestra.run.smithi092.stdout:(26/131): rbd-nbd-18.0.0-4313.ga02638f8.el8.x86 2.6 MB/s | 179 kB 00:00 2023-06-09T04:32:36.833 INFO:teuthology.orchestra.run.smithi092.stdout:(27/131): python3-rbd-18.0.0-4313.ga02638f8.el8 1.3 MB/s | 365 kB 00:00 2023-06-09T04:32:36.858 INFO:teuthology.orchestra.run.smithi092.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4313.g 276 kB/s | 23 kB 00:00 2023-06-09T04:32:37.017 INFO:teuthology.orchestra.run.smithi092.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4313.ga02638f 696 kB/s | 128 kB 00:00 2023-06-09T04:32:37.134 INFO:teuthology.orchestra.run.smithi092.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4313.ga0263 6.2 MB/s | 1.7 MB 00:00 2023-06-09T04:32:37.217 INFO:teuthology.orchestra.run.smithi092.stdout:(31/131): rbd-mirror-18.0.0-4313.ga02638f8.el8. 6.5 MB/s | 3.4 MB 00:00 2023-06-09T04:32:37.272 INFO:teuthology.orchestra.run.smithi092.stdout:(32/131): ceph-mgr-modules-core-18.0.0-4313.ga0 1.7 MB/s | 240 kB 00:00 2023-06-09T04:32:37.301 INFO:teuthology.orchestra.run.smithi092.stdout:(33/131): ceph-mgr-rook-18.0.0-4313.ga02638f8.e 567 kB/s | 47 kB 00:00 2023-06-09T04:32:37.326 INFO:teuthology.orchestra.run.smithi092.stdout:(34/131): ceph-prometheus-alerts-18.0.0-4313.ga 272 kB/s | 15 kB 00:00 2023-06-09T04:32:37.360 INFO:teuthology.orchestra.run.smithi092.stdout:(35/131): ceph-volume-18.0.0-4313.ga02638f8.el8 4.3 MB/s | 258 kB 00:00 2023-06-09T04:32:37.385 INFO:teuthology.orchestra.run.smithi092.stdout:(36/131): cephadm-18.0.0-4313.ga02638f8.el8.noa 2.7 MB/s | 160 kB 00:00 2023-06-09T04:32:37.410 INFO:teuthology.orchestra.run.smithi092.stdout:(37/131): cephfs-top-18.0.0-4313.ga02638f8.el8. 358 kB/s | 18 kB 00:00 2023-06-09T04:32:37.581 INFO:teuthology.orchestra.run.smithi123.stdout:(8/131): ceph-common-18.0.0-4313.ga02638f8.el8. 5.9 MB/s | 22 MB 00:03 2023-06-09T04:32:37.619 INFO:teuthology.orchestra.run.smithi092.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 282 kB/s | 59 kB 00:00 2023-06-09T04:32:37.656 INFO:teuthology.orchestra.run.smithi123.stdout:(9/131): ceph-selinux-18.0.0-4313.ga02638f8.el8 325 kB/s | 24 kB 00:00 2023-06-09T04:32:37.703 INFO:teuthology.orchestra.run.smithi092.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 966 kB/s | 306 kB 00:00 2023-06-09T04:32:37.728 INFO:teuthology.orchestra.run.smithi092.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 691 kB/s | 75 kB 00:00 2023-06-09T04:32:37.804 INFO:teuthology.orchestra.run.smithi092.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 580 kB/s | 44 kB 00:00 2023-06-09T04:32:37.837 INFO:teuthology.orchestra.run.smithi092.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-06-09T04:32:37.904 INFO:teuthology.orchestra.run.smithi092.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 496 kB/s | 33 kB 00:00 2023-06-09T04:32:37.929 INFO:teuthology.orchestra.run.smithi092.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-06-09T04:32:37.963 INFO:teuthology.orchestra.run.smithi092.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2023-06-09T04:32:38.005 INFO:teuthology.orchestra.run.smithi092.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-06-09T04:32:38.072 INFO:teuthology.orchestra.run.smithi092.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-06-09T04:32:38.097 INFO:teuthology.orchestra.run.smithi092.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2023-06-09T04:32:38.131 INFO:teuthology.orchestra.run.smithi092.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-06-09T04:32:38.148 INFO:teuthology.orchestra.run.smithi092.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-06-09T04:32:38.198 INFO:teuthology.orchestra.run.smithi092.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 976 kB/s | 49 kB 00:00 2023-06-09T04:32:38.257 INFO:teuthology.orchestra.run.smithi092.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-06-09T04:32:38.324 INFO:teuthology.orchestra.run.smithi092.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2023-06-09T04:32:38.340 INFO:teuthology.orchestra.run.smithi092.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 722 kB/s | 60 kB 00:00 2023-06-09T04:32:38.391 INFO:teuthology.orchestra.run.smithi092.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-06-09T04:32:38.416 INFO:teuthology.orchestra.run.smithi092.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2023-06-09T04:32:38.450 INFO:teuthology.orchestra.run.smithi092.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 574 kB/s | 34 kB 00:00 2023-06-09T04:32:38.483 INFO:teuthology.orchestra.run.smithi092.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-06-09T04:32:38.498 INFO:teuthology.orchestra.run.smithi123.stdout:(10/131): ceph-osd-18.0.0-4313.ga02638f8.el8.x8 9.2 MB/s | 18 MB 00:01 2023-06-09T04:32:38.517 INFO:teuthology.orchestra.run.smithi092.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 920 kB/s | 61 kB 00:00 2023-06-09T04:32:38.542 INFO:teuthology.orchestra.run.smithi092.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 319 kB/s | 19 kB 00:00 2023-06-09T04:32:38.576 INFO:teuthology.orchestra.run.smithi092.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-06-09T04:32:38.581 INFO:teuthology.orchestra.run.smithi123.stdout:(11/131): cephfs-mirror-18.0.0-4313.ga02638f8.e 3.0 MB/s | 258 kB 00:00 2023-06-09T04:32:38.609 INFO:teuthology.orchestra.run.smithi092.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 436 kB/s | 29 kB 00:00 2023-06-09T04:32:38.635 INFO:teuthology.orchestra.run.smithi092.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 469 kB/s | 27 kB 00:00 2023-06-09T04:32:38.648 INFO:teuthology.orchestra.run.smithi123.stdout:(12/131): libcephfs-devel-18.0.0-4313.ga02638f8 465 kB/s | 31 kB 00:00 2023-06-09T04:32:38.685 INFO:teuthology.orchestra.run.smithi092.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-06-09T04:32:38.710 INFO:teuthology.orchestra.run.smithi092.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-06-09T04:32:38.744 INFO:teuthology.orchestra.run.smithi092.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-06-09T04:32:38.777 INFO:teuthology.orchestra.run.smithi092.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-06-09T04:32:38.782 INFO:teuthology.orchestra.run.smithi123.stdout:(13/131): libcephfs2-18.0.0-4313.ga02638f8.el8. 5.7 MB/s | 771 kB 00:00 2023-06-09T04:32:38.803 INFO:teuthology.orchestra.run.smithi092.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-06-09T04:32:38.828 INFO:teuthology.orchestra.run.smithi092.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-06-09T04:32:38.857 INFO:teuthology.orchestra.run.smithi123.stdout:(14/131): libcephsqlite-18.0.0-4313.ga02638f8.e 1.8 MB/s | 137 kB 00:00 2023-06-09T04:32:38.967 INFO:teuthology.orchestra.run.smithi123.stdout:(15/131): librados-devel-18.0.0-4313.ga02638f8. 1.2 MB/s | 139 kB 00:00 2023-06-09T04:32:39.066 INFO:teuthology.orchestra.run.smithi123.stdout:(16/131): libradosstriper1-18.0.0-4313.ga02638f 4.7 MB/s | 473 kB 00:00 2023-06-09T04:32:39.137 INFO:teuthology.orchestra.run.smithi092.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 44 MB/s | 14 MB 00:00 2023-06-09T04:32:39.179 INFO:teuthology.orchestra.run.smithi092.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 4.3 MB/s | 1.7 MB 00:00 2023-06-09T04:32:39.204 INFO:teuthology.orchestra.run.smithi092.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-06-09T04:32:39.246 INFO:teuthology.orchestra.run.smithi092.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 7.0 MB/s | 259 kB 00:00 2023-06-09T04:32:39.271 INFO:teuthology.orchestra.run.smithi092.stdout:(74/131): python3-pytz-2017.2-9.el8.noarch.rpm 588 kB/s | 54 kB 00:00 2023-06-09T04:32:39.297 INFO:teuthology.orchestra.run.smithi092.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-06-09T04:32:39.322 INFO:teuthology.orchestra.run.smithi092.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 770 kB/s | 39 kB 00:00 2023-06-09T04:32:39.356 INFO:teuthology.orchestra.run.smithi092.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-06-09T04:32:39.389 INFO:teuthology.orchestra.run.smithi092.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-06-09T04:32:39.406 INFO:teuthology.orchestra.run.smithi092.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-06-09T04:32:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-06-09T04:32:39.465 INFO:teuthology.orchestra.run.smithi092.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-06-09T04:32:39.499 INFO:teuthology.orchestra.run.smithi092.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 801 kB/s | 27 kB 00:00 2023-06-09T04:32:39.540 INFO:teuthology.orchestra.run.smithi092.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-09T04:32:39.616 INFO:teuthology.orchestra.run.smithi092.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-06-09T04:32:39.633 INFO:teuthology.orchestra.run.smithi092.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 478 kB/s | 44 kB 00:00 2023-06-09T04:32:39.675 INFO:teuthology.orchestra.run.smithi092.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-06-09T04:32:39.716 INFO:teuthology.orchestra.run.smithi123.stdout:(17/131): librgw2-18.0.0-4313.ga02638f8.el8.x86 10 MB/s | 6.5 MB 00:00 2023-06-09T04:32:39.717 INFO:teuthology.orchestra.run.smithi092.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-06-09T04:32:39.775 INFO:teuthology.orchestra.run.smithi092.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 23 MB/s | 3.7 MB 00:00 2023-06-09T04:32:39.792 INFO:teuthology.orchestra.run.smithi092.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 853 kB/s | 64 kB 00:00 2023-06-09T04:32:39.809 INFO:teuthology.orchestra.run.smithi092.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.7 MB/s | 126 kB 00:00 2023-06-09T04:32:39.817 INFO:teuthology.orchestra.run.smithi123.stdout:(18/131): python3-ceph-argparse-18.0.0-4313.ga0 448 kB/s | 45 kB 00:00 2023-06-09T04:32:39.826 INFO:teuthology.orchestra.run.smithi092.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 593 kB/s | 20 kB 00:00 2023-06-09T04:32:39.851 INFO:teuthology.orchestra.run.smithi092.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-06-09T04:32:39.885 INFO:teuthology.orchestra.run.smithi092.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-09T04:32:39.900 INFO:teuthology.orchestra.run.smithi123.stdout:(19/131): python3-ceph-common-18.0.0-4313.ga026 1.4 MB/s | 119 kB 00:00 2023-06-09T04:32:39.927 INFO:teuthology.orchestra.run.smithi092.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-06-09T04:32:39.976 INFO:teuthology.orchestra.run.smithi123.stdout:(20/131): python3-cephfs-18.0.0-4313.ga02638f8. 2.6 MB/s | 197 kB 00:00 2023-06-09T04:32:40.019 INFO:teuthology.orchestra.run.smithi092.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-06-09T04:32:40.059 INFO:teuthology.orchestra.run.smithi123.stdout:(21/131): python3-rados-18.0.0-4313.ga02638f8.e 4.4 MB/s | 370 kB 00:00 2023-06-09T04:32:40.061 INFO:teuthology.orchestra.run.smithi092.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-06-09T04:32:40.136 INFO:teuthology.orchestra.run.smithi092.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-06-09T04:32:40.151 INFO:teuthology.orchestra.run.smithi123.stdout:(22/131): python3-rbd-18.0.0-4313.ga02638f8.el8 3.9 MB/s | 365 kB 00:00 2023-06-09T04:32:40.162 INFO:teuthology.orchestra.run.smithi092.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-06-09T04:32:40.212 INFO:teuthology.orchestra.run.smithi092.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-06-09T04:32:40.243 INFO:teuthology.orchestra.run.smithi123.stdout:(23/131): python3-rgw-18.0.0-4313.ga02638f8.el8 1.1 MB/s | 107 kB 00:00 2023-06-09T04:32:40.296 INFO:teuthology.orchestra.run.smithi092.stdout:(100/131): postgresql-server-10.21-2.module+el8 32 MB/s | 5.1 MB 00:00 2023-06-09T04:32:40.318 INFO:teuthology.orchestra.run.smithi123.stdout:(24/131): rbd-fuse-18.0.0-4313.ga02638f8.el8.x8 1.2 MB/s | 94 kB 00:00 2023-06-09T04:32:40.338 INFO:teuthology.orchestra.run.smithi092.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-06-09T04:32:40.363 INFO:teuthology.orchestra.run.smithi092.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 701 kB/s | 47 kB 00:00 2023-06-09T04:32:40.388 INFO:teuthology.orchestra.run.smithi092.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-06-09T04:32:40.413 INFO:teuthology.orchestra.run.smithi092.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-06-09T04:32:40.439 INFO:teuthology.orchestra.run.smithi092.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-06-09T04:32:40.464 INFO:teuthology.orchestra.run.smithi092.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 957 kB/s | 48 kB 00:00 2023-06-09T04:32:40.497 INFO:teuthology.orchestra.run.smithi092.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-06-09T04:32:40.523 INFO:teuthology.orchestra.run.smithi092.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 727 kB/s | 43 kB 00:00 2023-06-09T04:32:40.548 INFO:teuthology.orchestra.run.smithi092.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-06-09T04:32:40.573 INFO:teuthology.orchestra.run.smithi092.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-06-09T04:32:40.607 INFO:teuthology.orchestra.run.smithi092.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-06-09T04:32:40.691 INFO:teuthology.orchestra.run.smithi092.stdout:(112/131): ceph-mgr-diskprediction-local-18.0.0 2.0 MB/s | 7.4 MB 00:03 2023-06-09T04:32:40.716 INFO:teuthology.orchestra.run.smithi092.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 131 kB/s | 19 kB 00:00 2023-06-09T04:32:40.719 INFO:teuthology.orchestra.run.smithi123.stdout:(25/131): rbd-mirror-18.0.0-4313.ga02638f8.el8. 8.6 MB/s | 3.4 MB 00:00 2023-06-09T04:32:40.741 INFO:teuthology.orchestra.run.smithi092.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-06-09T04:32:40.766 INFO:teuthology.orchestra.run.smithi092.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 400 kB/s | 18 kB 00:00 2023-06-09T04:32:40.785 INFO:teuthology.orchestra.run.smithi123.stdout:(26/131): rbd-nbd-18.0.0-4313.ga02638f8.el8.x86 2.6 MB/s | 179 kB 00:00 2023-06-09T04:32:40.800 INFO:teuthology.orchestra.run.smithi092.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-06-09T04:32:40.825 INFO:teuthology.orchestra.run.smithi092.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-06-09T04:32:40.859 INFO:teuthology.orchestra.run.smithi092.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-06-09T04:32:40.869 INFO:teuthology.orchestra.run.smithi123.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4313.g 277 kB/s | 23 kB 00:00 2023-06-09T04:32:40.884 INFO:teuthology.orchestra.run.smithi092.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-06-09T04:32:40.909 INFO:teuthology.orchestra.run.smithi092.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-06-09T04:32:40.935 INFO:teuthology.orchestra.run.smithi092.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-06-09T04:32:40.945 INFO:teuthology.orchestra.run.smithi123.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4313.ga02638f 1.7 MB/s | 128 kB 00:00 2023-06-09T04:32:40.960 INFO:teuthology.orchestra.run.smithi092.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-06-09T04:32:40.985 INFO:teuthology.orchestra.run.smithi092.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-06-09T04:32:41.011 INFO:teuthology.orchestra.run.smithi092.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 766 kB/s | 39 kB 00:00 2023-06-09T04:32:41.044 INFO:teuthology.orchestra.run.smithi092.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-06-09T04:32:41.069 INFO:teuthology.orchestra.run.smithi092.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-06-09T04:32:41.095 INFO:teuthology.orchestra.run.smithi092.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-06-09T04:32:41.145 INFO:teuthology.orchestra.run.smithi092.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.7 MB/s | 2.3 MB 00:00 2023-06-09T04:32:41.195 INFO:teuthology.orchestra.run.smithi092.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-06-09T04:32:41.395 INFO:teuthology.orchestra.run.smithi123.stdout:(29/131): ceph-test-18.0.0-4313.ga02638f8.el8.x 11 MB/s | 41 MB 00:03 2023-06-09T04:32:41.437 INFO:teuthology.orchestra.run.smithi092.stdout:(130/131): librados2-18.0.0-4313.ga02638f8.el8. 10 MB/s | 3.7 MB 00:00 2023-06-09T04:32:41.445 INFO:teuthology.orchestra.run.smithi123.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4313.ga0263 3.4 MB/s | 1.7 MB 00:00 2023-06-09T04:32:41.545 INFO:teuthology.orchestra.run.smithi123.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4313.ga0 2.4 MB/s | 240 kB 00:00 2023-06-09T04:32:41.629 INFO:teuthology.orchestra.run.smithi123.stdout:(32/131): ceph-mgr-rook-18.0.0-4313.ga02638f8.e 570 kB/s | 47 kB 00:00 2023-06-09T04:32:41.688 INFO:teuthology.orchestra.run.smithi123.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4313.ga 249 kB/s | 15 kB 00:00 2023-06-09T04:32:41.763 INFO:teuthology.orchestra.run.smithi123.stdout:(34/131): ceph-volume-18.0.0-4313.ga02638f8.el8 3.4 MB/s | 258 kB 00:00 2023-06-09T04:32:41.846 INFO:teuthology.orchestra.run.smithi123.stdout:(35/131): cephadm-18.0.0-4313.ga02638f8.el8.noa 1.9 MB/s | 160 kB 00:00 2023-06-09T04:32:41.905 INFO:teuthology.orchestra.run.smithi123.stdout:(36/131): cephfs-top-18.0.0-4313.ga02638f8.el8. 307 kB/s | 18 kB 00:00 2023-06-09T04:32:42.064 INFO:teuthology.orchestra.run.smithi123.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-06-09T04:32:42.137 INFO:teuthology.orchestra.run.smithi092.stdout:(131/131): librbd1-18.0.0-4313.ga02638f8.el8.x8 3.4 MB/s | 3.6 MB 00:01 2023-06-09T04:32:42.140 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-06-09T04:32:42.140 INFO:teuthology.orchestra.run.smithi092.stdout:Total 17 MB/s | 197 MB 00:11 2023-06-09T04:32:42.189 INFO:teuthology.orchestra.run.smithi123.stdout:(38/131): ceph-radosgw-18.0.0-4313.ga02638f8.el 2.4 MB/s | 13 MB 00:05 2023-06-09T04:32:42.231 INFO:teuthology.orchestra.run.smithi123.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 941 kB/s | 306 kB 00:00 2023-06-09T04:32:42.314 INFO:teuthology.orchestra.run.smithi123.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 235 kB/s | 59 kB 00:00 2023-06-09T04:32:42.390 INFO:teuthology.orchestra.run.smithi123.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2023-06-09T04:32:42.415 INFO:teuthology.orchestra.run.smithi123.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 332 kB/s | 75 kB 00:00 2023-06-09T04:32:42.440 INFO:teuthology.orchestra.run.smithi123.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 347 kB/s | 44 kB 00:00 2023-06-09T04:32:42.474 INFO:teuthology.orchestra.run.smithi123.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-06-09T04:32:42.549 INFO:teuthology.orchestra.run.smithi123.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2023-06-09T04:32:42.575 INFO:teuthology.orchestra.run.smithi123.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 122 kB/s | 16 kB 00:00 2023-06-09T04:32:42.608 INFO:teuthology.orchestra.run.smithi123.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-06-09T04:32:42.633 INFO:teuthology.orchestra.run.smithi123.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-06-09T04:32:42.667 INFO:teuthology.orchestra.run.smithi123.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-06-09T04:32:42.701 INFO:teuthology.orchestra.run.smithi123.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2023-06-09T04:32:42.759 INFO:teuthology.orchestra.run.smithi123.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-06-09T04:32:42.784 INFO:teuthology.orchestra.run.smithi123.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 583 kB/s | 49 kB 00:00 2023-06-09T04:32:42.826 INFO:teuthology.orchestra.run.smithi123.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 879 kB/s | 59 kB 00:00 2023-06-09T04:32:42.852 INFO:teuthology.orchestra.run.smithi123.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 902 kB/s | 60 kB 00:00 2023-06-09T04:32:42.894 INFO:teuthology.orchestra.run.smithi123.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 5.2 MB/s | 1.2 MB 00:00 2023-06-09T04:32:42.919 INFO:teuthology.orchestra.run.smithi123.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-06-09T04:32:42.956 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-06-09T04:32:42.961 INFO:teuthology.orchestra.run.smithi123.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2023-06-09T04:32:42.986 INFO:teuthology.orchestra.run.smithi123.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-06-09T04:32:43.028 INFO:teuthology.orchestra.run.smithi123.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 921 kB/s | 61 kB 00:00 2023-06-09T04:32:43.053 INFO:teuthology.orchestra.run.smithi123.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2023-06-09T04:32:43.070 INFO:teuthology.orchestra.run.smithi123.stdout:(61/131): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-06-09T04:32:43.077 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-06-09T04:32:43.078 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-06-09T04:32:43.118 INFO:teuthology.orchestra.run.smithi123.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 267 kB/s | 24 kB 00:00 2023-06-09T04:32:43.146 INFO:teuthology.orchestra.run.smithi123.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 317 kB/s | 29 kB 00:00 2023-06-09T04:32:43.171 INFO:teuthology.orchestra.run.smithi123.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-06-09T04:32:43.196 INFO:teuthology.orchestra.run.smithi123.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.2 MB/s | 255 kB 00:00 2023-06-09T04:32:43.239 INFO:teuthology.orchestra.run.smithi123.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 915 kB/s | 61 kB 00:00 2023-06-09T04:32:43.264 INFO:teuthology.orchestra.run.smithi123.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-06-09T04:32:43.289 INFO:teuthology.orchestra.run.smithi123.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 925 kB/s | 85 kB 00:00 2023-06-09T04:32:43.314 INFO:teuthology.orchestra.run.smithi123.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2023-06-09T04:32:43.339 INFO:teuthology.orchestra.run.smithi123.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-06-09T04:32:43.390 INFO:teuthology.orchestra.run.smithi123.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 622 kB/s | 47 kB 00:00 2023-06-09T04:32:43.415 INFO:teuthology.orchestra.run.smithi123.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.8 MB/s | 200 kB 00:00 2023-06-09T04:32:43.440 INFO:teuthology.orchestra.run.smithi123.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-06-09T04:32:43.466 INFO:teuthology.orchestra.run.smithi123.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-06-09T04:32:43.491 INFO:teuthology.orchestra.run.smithi123.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 957 kB/s | 48 kB 00:00 2023-06-09T04:32:43.516 INFO:teuthology.orchestra.run.smithi123.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-09T04:32:43.542 INFO:teuthology.orchestra.run.smithi123.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 850 kB/s | 43 kB 00:00 2023-06-09T04:32:43.567 INFO:teuthology.orchestra.run.smithi123.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-06-09T04:32:43.592 INFO:teuthology.orchestra.run.smithi123.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-06-09T04:32:43.626 INFO:teuthology.orchestra.run.smithi123.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-06-09T04:32:43.651 INFO:teuthology.orchestra.run.smithi123.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-06-09T04:32:43.695 INFO:teuthology.orchestra.run.smithi123.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.2 MB/s | 155 kB 00:00 2023-06-09T04:32:43.735 INFO:teuthology.orchestra.run.smithi123.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 6.5 MB/s | 559 kB 00:00 2023-06-09T04:32:43.760 INFO:teuthology.orchestra.run.smithi123.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 278 kB/s | 18 kB 00:00 2023-06-09T04:32:43.794 INFO:teuthology.orchestra.run.smithi123.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-06-09T04:32:43.819 INFO:teuthology.orchestra.run.smithi123.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-06-09T04:32:43.861 INFO:teuthology.orchestra.run.smithi123.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-06-09T04:32:43.903 INFO:teuthology.orchestra.run.smithi123.stdout:(88/131): thrift-0.13.0-2.el8.x86_64.rpm 2.7 MB/s | 1.7 MB 00:00 2023-06-09T04:32:43.920 INFO:teuthology.orchestra.run.smithi123.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-06-09T04:32:43.979 INFO:teuthology.orchestra.run.smithi123.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-06-09T04:32:43.996 INFO:teuthology.orchestra.run.smithi123.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 650 kB/s | 60 kB 00:00 2023-06-09T04:32:44.021 INFO:teuthology.orchestra.run.smithi123.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.0 MB/s | 98 kB 00:00 2023-06-09T04:32:44.046 INFO:teuthology.orchestra.run.smithi123.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 858 kB/s | 58 kB 00:00 2023-06-09T04:32:44.071 INFO:teuthology.orchestra.run.smithi123.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 511 kB/s | 39 kB 00:00 2023-06-09T04:32:44.096 INFO:teuthology.orchestra.run.smithi123.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 171 kB 00:00 2023-06-09T04:32:44.122 INFO:teuthology.orchestra.run.smithi123.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-06-09T04:32:44.154 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-06-09T04:32:44.155 INFO:teuthology.orchestra.run.smithi123.stdout:(97/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.7 MB/s | 644 kB 00:00 2023-06-09T04:32:44.158 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-06-09T04:32:44.172 INFO:teuthology.orchestra.run.smithi123.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.1 MB/s | 110 kB 00:00 2023-06-09T04:32:44.198 INFO:teuthology.orchestra.run.smithi123.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-06-09T04:32:44.223 INFO:teuthology.orchestra.run.smithi123.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-06-09T04:32:44.273 INFO:teuthology.orchestra.run.smithi123.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-06-09T04:32:44.307 INFO:teuthology.orchestra.run.smithi123.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 465 kB/s | 39 kB 00:00 2023-06-09T04:32:44.341 INFO:teuthology.orchestra.run.smithi123.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-06-09T04:32:44.376 INFO:teuthology.orchestra.run.smithi123.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.8 MB/s | 538 kB 00:00 2023-06-09T04:32:44.400 INFO:teuthology.orchestra.run.smithi123.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-06-09T04:32:44.425 INFO:teuthology.orchestra.run.smithi123.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-06-09T04:32:44.459 INFO:teuthology.orchestra.run.smithi123.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-06-09T04:32:44.492 INFO:teuthology.orchestra.run.smithi123.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 797 kB/s | 27 kB 00:00 2023-06-09T04:32:44.534 INFO:teuthology.orchestra.run.smithi123.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-09T04:32:44.567 INFO:teuthology.orchestra.run.smithi123.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-06-09T04:32:44.651 INFO:teuthology.orchestra.run.smithi123.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-06-09T04:32:44.702 INFO:teuthology.orchestra.run.smithi123.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-06-09T04:32:44.743 INFO:teuthology.orchestra.run.smithi123.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-06-09T04:32:44.810 INFO:teuthology.orchestra.run.smithi123.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-06-09T04:32:44.852 INFO:teuthology.orchestra.run.smithi123.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 589 kB/s | 64 kB 00:00 2023-06-09T04:32:44.877 INFO:teuthology.orchestra.run.smithi123.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.8 MB/s | 126 kB 00:00 2023-06-09T04:32:44.894 INFO:teuthology.orchestra.run.smithi123.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 479 kB/s | 20 kB 00:00 2023-06-09T04:32:44.920 INFO:teuthology.orchestra.run.smithi123.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-06-09T04:32:44.953 INFO:teuthology.orchestra.run.smithi123.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-09T04:32:45.095 INFO:teuthology.orchestra.run.smithi123.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 14 MB/s | 14 MB 00:00 2023-06-09T04:32:45.120 INFO:teuthology.orchestra.run.smithi123.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-06-09T04:32:45.162 INFO:teuthology.orchestra.run.smithi123.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-06-09T04:32:45.204 INFO:teuthology.orchestra.run.smithi123.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-06-09T04:32:45.288 INFO:teuthology.orchestra.run.smithi123.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-06-09T04:32:45.363 INFO:teuthology.orchestra.run.smithi123.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-06-09T04:32:45.480 INFO:teuthology.orchestra.run.smithi123.stdout:(126/131): postgresql-contrib-10.21-2.module+el 4.1 MB/s | 808 kB 00:00 2023-06-09T04:32:45.697 INFO:teuthology.orchestra.run.smithi123.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 4.5 MB/s | 1.5 MB 00:00 2023-06-09T04:32:45.781 INFO:teuthology.orchestra.run.smithi123.stdout:(128/131): postgresql-server-10.21-2.module+el8 8.8 MB/s | 5.1 MB 00:00 2023-06-09T04:32:45.839 INFO:teuthology.orchestra.run.smithi123.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-06-09T04:32:46.256 INFO:teuthology.orchestra.run.smithi123.stdout:(130/131): librbd1-18.0.0-4313.ga02638f8.el8.x8 6.4 MB/s | 3.6 MB 00:00 2023-06-09T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stdout:(131/131): librados2-18.0.0-4313.ga02638f8.el8. 3.0 MB/s | 3.7 MB 00:01 2023-06-09T04:32:46.701 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-06-09T04:32:46.701 INFO:teuthology.orchestra.run.smithi123.stdout:Total 15 MB/s | 197 MB 00:12 2023-06-09T04:32:47.523 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-06-09T04:32:47.641 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-06-09T04:32:47.642 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-06-09T04:32:48.730 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-06-09T04:32:48.734 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-06-09T04:32:48.911 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-06-09T04:32:49.093 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-09T04:32:49.272 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-09T04:32:49.477 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-06-09T04:32:49.645 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-09T04:32:49.769 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-09T04:32:49.970 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-09T04:32:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-09T04:32:50.578 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-09T04:32:50.979 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-09T04:32:51.194 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-09T04:32:51.320 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-09T04:32:51.494 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-09T04:32:51.596 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-09T04:32:51.770 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 9/134 2023-06-09T04:32:51.998 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-09T04:32:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-09T04:32:52.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-09T04:32:52.865 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-09T04:32:53.069 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-09T04:32:53.173 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-09T04:32:53.336 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-09T04:32:53.418 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-09T04:32:53.552 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-09T04:32:53.787 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-09T04:32:53.876 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-06-09T04:32:53.935 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-09T04:32:54.057 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-09T04:32:54.282 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-09T04:32:54.459 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-09T04:32:54.554 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-09T04:32:54.585 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-09T04:32:54.719 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-09T04:32:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-09T04:32:55.213 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-09T04:32:55.320 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-09T04:32:55.442 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 5/134 2023-06-09T04:32:55.527 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-06-09T04:32:55.851 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 5/134 2023-06-09T04:32:56.059 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-09T04:32:56.198 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-09T04:32:56.238 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-06-09T04:32:56.426 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-09T04:32:56.500 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-06-09T04:32:56.536 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-09T04:32:56.736 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-06-09T04:32:56.814 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-09T04:32:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-06-09T04:32:57.029 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-09T04:32:57.116 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-06-09T04:32:57.304 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-09T04:32:57.341 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-06-09T04:32:57.511 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-09T04:32:57.545 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-09T04:32:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 26/134 2023-06-09T04:32:57.862 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 27/134 2023-06-09T04:32:57.926 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-09T04:32:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-09T04:32:58.392 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-09T04:32:58.501 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-09T04:32:58.683 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-09T04:32:58.770 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 28/134 2023-06-09T04:32:58.879 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-09T04:32:59.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-09T04:32:59.240 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-09T04:32:59.240 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-06-09T04:32:59.449 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-09T04:32:59.504 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-06-09T04:32:59.599 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-09T04:32:59.741 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-09T04:32:59.797 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-06-09T04:32:59.900 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-09T04:33:00.009 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-09T04:33:00.504 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-09T04:33:00.691 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-09T04:33:01.278 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-09T04:33:01.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-09T04:33:01.766 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-09T04:33:02.086 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-06-09T04:33:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-09T04:33:02.286 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-09T04:33:02.469 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-09T04:33:02.608 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-09T04:33:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 36/134 2023-06-09T04:33:02.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-09T04:33:02.832 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 37/134 2023-06-09T04:33:02.995 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-06-09T04:33:03.025 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-09T04:33:03.160 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-06-09T04:33:03.197 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 26/134 2023-06-09T04:33:03.319 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-06-09T04:33:03.358 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 27/134 2023-06-09T04:33:03.453 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-06-09T04:33:03.595 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 28/134 2023-06-09T04:33:03.629 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-06-09T04:33:03.782 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-09T04:33:03.835 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-06-09T04:33:04.167 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-06-09T04:33:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-06-09T04:33:04.918 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-06-09T04:33:05.129 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-06-09T04:33:05.355 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-06-09T04:33:05.534 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-09T04:33:05.666 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-06-09T04:33:05.932 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-06-09T04:33:06.125 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-06-09T04:33:06.329 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-06-09T04:33:06.504 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-06-09T04:33:06.559 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-09T04:33:06.685 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-06-09T04:33:06.910 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-09T04:33:06.993 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-06-09T04:33:07.184 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-09T04:33:07.253 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-06-09T04:33:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-09T04:33:07.479 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-06-09T04:33:07.626 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-09T04:33:07.723 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-06-09T04:33:07.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-09T04:33:07.906 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-06-09T04:33:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 36/134 2023-06-09T04:33:08.072 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-06-09T04:33:08.188 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 37/134 2023-06-09T04:33:08.257 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-06-09T04:33:08.326 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-06-09T04:33:08.347 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-09T04:33:08.530 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-09T04:33:08.599 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-06-09T04:33:08.732 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-09T04:33:08.801 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-09T04:33:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-06-09T04:33:08.991 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-09T04:33:09.004 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-09T04:33:09.072 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-09T04:33:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-09T04:33:09.280 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-06-09T04:33:09.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-09T04:33:09.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-06-09T04:33:09.479 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-09T04:33:09.599 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-09T04:33:09.694 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-09T04:33:09.770 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-09T04:33:09.919 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-09T04:33:10.072 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-09T04:33:10.082 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-09T04:33:10.291 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-09T04:33:10.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-09T04:33:10.478 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-09T04:33:10.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-09T04:33:10.643 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-09T04:33:10.651 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-09T04:33:10.850 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-09T04:33:10.853 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-09T04:33:11.070 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-09T04:33:11.085 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-09T04:33:11.268 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-09T04:33:11.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-09T04:33:11.460 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-09T04:33:11.596 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-09T04:33:11.724 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-09T04:33:11.794 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-09T04:33:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-09T04:33:12.010 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-09T04:33:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-09T04:33:12.234 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-09T04:33:12.451 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-09T04:33:12.477 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-09T04:33:12.680 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-09T04:33:12.814 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-09T04:33:12.855 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-09T04:33:13.022 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-09T04:33:13.065 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-09T04:33:13.255 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-09T04:33:13.474 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-09T04:33:13.629 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-09T04:33:13.818 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-09T04:33:14.124 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-09T04:33:14.125 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-09T04:33:14.368 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-09T04:33:14.385 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-09T04:33:14.602 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-09T04:33:14.623 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-09T04:33:14.832 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-09T04:33:14.907 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-09T04:33:15.013 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-09T04:33:15.175 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-09T04:33:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-09T04:33:15.399 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-09T04:33:15.442 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-09T04:33:15.679 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-09T04:33:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-09T04:33:16.008 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-09T04:33:16.167 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-09T04:33:16.215 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-09T04:33:16.331 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-09T04:33:16.405 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-09T04:33:16.408 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-09T04:33:16.564 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 93/134 2023-06-09T04:33:16.582 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-09T04:33:16.750 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-09T04:33:16.895 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-09T04:33:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-09T04:33:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-09T04:33:17.503 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-09T04:33:17.778 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-09T04:33:18.046 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-09T04:33:18.228 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-09T04:33:18.485 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-09T04:33:19.559 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-09T04:33:19.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-09T04:33:20.037 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-09T04:33:20.159 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-09T04:33:20.330 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-09T04:33:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-09T04:33:20.672 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-09T04:33:20.878 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-09T04:33:21.254 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-09T04:33:21.521 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-09T04:33:21.606 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-09T04:33:21.663 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-09T04:33:21.729 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-09T04:33:21.855 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-09T04:33:21.918 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-09T04:33:22.080 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 93/134 2023-06-09T04:33:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-09T04:33:25.913 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-09T04:33:26.501 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-09T04:33:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-09T04:33:26.845 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-09T04:33:26.987 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-09T04:33:40.491 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-09T04:33:40.491 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2023-06-09T04:33:40.491 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2023-06-09T04:33:40.491 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-06-09T04:33:40.492 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /run 2023-06-09T04:33:40.492 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2023-06-09T04:33:40.492 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2023-06-09T04:33:40.492 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2023-06-09T04:33:40.492 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2023-06-09T04:33:40.493 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-06-09T04:33:40.493 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:33:40.859 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-09T04:33:42.447 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-09T04:33:42.608 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-09T04:33:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-09T04:33:42.975 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-09T04:33:43.424 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-09T04:33:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-09T04:33:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-09T04:33:45.289 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-09T04:33:45.485 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-09T04:33:45.613 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 102/134 2023-06-09T04:33:45.645 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-09T04:33:45.645 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2023-06-09T04:33:45.645 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2023-06-09T04:33:45.646 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-06-09T04:33:45.646 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /run 2023-06-09T04:33:45.646 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2023-06-09T04:33:45.646 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2023-06-09T04:33:45.646 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2023-06-09T04:33:45.646 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2023-06-09T04:33:45.646 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-06-09T04:33:45.647 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:33:45.788 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 103/134 2023-06-09T04:33:45.974 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-09T04:33:46.009 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-09T04:33:46.185 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-09T04:33:46.432 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-09T04:33:47.390 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-09T04:33:47.549 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-09T04:33:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-09T04:33:47.639 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-09T04:33:47.799 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-09T04:33:48.009 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 107/134 2023-06-09T04:33:48.019 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-09T04:33:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-09T04:33:48.231 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-09T04:33:48.267 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-09T04:33:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-09T04:33:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-09T04:33:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-09T04:33:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-09T04:33:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-09T04:33:49.707 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 110/134 2023-06-09T04:33:49.957 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-09T04:33:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-09T04:33:50.148 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-09T04:33:50.190 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-09T04:33:50.348 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-09T04:33:50.439 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-09T04:33:50.659 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-09T04:33:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-09T04:33:50.789 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 102/134 2023-06-09T04:33:50.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-09T04:33:50.963 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 103/134 2023-06-09T04:33:51.183 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-09T04:33:51.395 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-09T04:33:51.673 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-09T04:33:52.635 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-09T04:33:52.800 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-09T04:33:52.889 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-09T04:33:53.243 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 107/134 2023-06-09T04:33:53.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-09T04:33:53.499 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-09T04:33:53.650 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-09T04:33:53.661 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-09T04:33:54.749 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-09T04:33:54.912 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 110/134 2023-06-09T04:33:55.035 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 115/134 2023-06-09T04:33:55.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-09T04:33:55.172 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-09T04:33:55.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-06-09T04:33:55.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-09T04:33:55.374 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-09T04:33:55.414 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-06-09T04:33:55.533 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-09T04:33:55.880 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-09T04:33:56.049 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-09T04:33:56.330 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-06-09T04:33:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-06-09T04:33:56.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 119/134 2023-06-09T04:33:56.992 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-09T04:33:57.150 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 121/134 2023-06-09T04:33:57.289 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-09T04:33:59.442 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T04:33:59.704 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T04:33:59.721 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T04:33:59.883 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 124/134 2023-06-09T04:34:00.009 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-09T04:34:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 125/134 2023-06-09T04:34:00.433 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 126/134 2023-06-09T04:34:00.731 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-09T04:34:00.863 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 115/134 2023-06-09T04:34:00.870 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-09T04:34:00.879 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-09T04:34:01.061 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-06-09T04:34:01.197 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-09T04:34:01.237 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-06-09T04:34:01.322 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-06-09T04:34:01.392 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 117/134 2023-06-09T04:34:01.529 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-06-09T04:34:01.541 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-06-09T04:34:01.542 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T04:34:01.662 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-09T04:34:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T04:34:01.715 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T04:34:01.848 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-09T04:34:01.952 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T04:34:02.038 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-09T04:34:02.161 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T04:34:02.327 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-09T04:34:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T04:34:02.438 INFO:teuthology.orchestra.run.smithi092.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-09T04:34:02.438 INFO:teuthology.orchestra.run.smithi092.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-09T04:34:02.438 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:02.502 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 121/134 2023-06-09T04:34:02.641 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-09T04:34:02.649 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 1/134 2023-06-09T04:34:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2/134 2023-06-09T04:34:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 3/134 2023-06-09T04:34:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 4/134 2023-06-09T04:34:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 5/134 2023-06-09T04:34:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-09T04:34:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-09T04:34:02.650 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-09T04:34:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 9/134 2023-06-09T04:34:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 10/134 2023-06-09T04:34:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 11/134 2023-06-09T04:34:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 12/134 2023-06-09T04:34:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 13/134 2023-06-09T04:34:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 14/134 2023-06-09T04:34:02.651 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 15/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 16/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 17/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 18/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 19/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 20/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 21/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 22/134 2023-06-09T04:34:02.652 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 23/134 2023-06-09T04:34:02.653 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 24/134 2023-06-09T04:34:02.659 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 25/134 2023-06-09T04:34:02.660 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 26/134 2023-06-09T04:34:02.660 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 27/134 2023-06-09T04:34:02.660 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 28/134 2023-06-09T04:34:02.660 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 29/134 2023-06-09T04:34:02.660 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 30/134 2023-06-09T04:34:02.660 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 31/134 2023-06-09T04:34:02.660 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 32/134 2023-06-09T04:34:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 33/134 2023-06-09T04:34:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 34/134 2023-06-09T04:34:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 35/134 2023-06-09T04:34:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 36/134 2023-06-09T04:34:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 37/134 2023-06-09T04:34:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 38/134 2023-06-09T04:34:02.661 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-09T04:34:02.662 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-09T04:34:02.663 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-09T04:34:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-09T04:34:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-09T04:34:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-09T04:34:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-09T04:34:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-09T04:34:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-09T04:34:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-09T04:34:02.665 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-06-09T04:34:02.666 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-06-09T04:34:02.667 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-06-09T04:34:02.668 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-06-09T04:34:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-06-09T04:34:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-06-09T04:34:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-06-09T04:34:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-06-09T04:34:02.669 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-06-09T04:34:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-06-09T04:34:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-06-09T04:34:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-06-09T04:34:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-06-09T04:34:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-06-09T04:34:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-06-09T04:34:02.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-06-09T04:34:02.671 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-06-09T04:34:02.672 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-09T04:34:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-06-09T04:34:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-06-09T04:34:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-06-09T04:34:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-06-09T04:34:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-06-09T04:34:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-09T04:34:02.673 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 129/134 2023-06-09T04:34:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-09T04:34:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 131/134 2023-06-09T04:34:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T04:34:02.674 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-09T04:34:04.770 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T04:34:05.064 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T04:34:05.081 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T04:34:05.265 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T04:34:05.265 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:46 ago on Fri 09 Jun 2023 04:32:19 AM UTC. 2023-06-09T04:34:05.267 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 124/134 2023-06-09T04:34:05.506 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 125/134 2023-06-09T04:34:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 126/134 2023-06-09T04:34:06.311 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-09T04:34:06.489 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-09T04:34:06.805 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-09T04:34:06.818 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-09T04:34:06.981 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-09T04:34:06.989 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-09T04:34:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-09T04:34:07.125 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T04:34:07.367 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T04:34:07.367 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T04:34:07.612 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T04:34:07.778 INFO:teuthology.orchestra.run.smithi092.stdout:Installed products updated. 2023-06-09T04:34:07.812 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T04:34:08.001 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:08.002 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-06-09T04:34:08.002 INFO:teuthology.orchestra.run.smithi092.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-09T04:34:08.002 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.002 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.002 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-06-09T04:34:08.002 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-09T04:34:08.003 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.1-7.el8.noarch 2023-06-09T04:34:08.003 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.0.4-10.el8.x86_64 2023-06-09T04:34:08.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.1-9.el8.x86_64 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.6-3.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-09T04:34:08.007 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-09T04:34:08.008 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-09T04:34:08.008 INFO:teuthology.orchestra.run.smithi092.stdout: libpq-13.5-1.el8.x86_64 2023-06-09T04:34:08.008 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-09T04:34:08.008 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-09T04:34:08.008 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.008 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.008 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-09T04:34:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-09T04:34:08.010 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T04:34:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T04:34:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T04:34:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-09T04:34:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-09T04:34:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-09T04:34:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-09T04:34:08.011 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-09T04:34:08.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-09T04:34:08.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-09T04:34:08.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-09T04:34:08.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-09T04:34:08.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-09T04:34:08.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-09T04:34:08.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-09T04:34:08.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-09T04:34:08.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-2.6-1.el8.noarch 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-09T04:34:08.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-09T04:34:08.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-09T04:34:08.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-09T04:34:08.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-09T04:34:08.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-09T04:34:08.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:08.018 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-09T04:34:08.019 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-09T04:34:08.019 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-09T04:34:08.019 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-09T04:34:08.019 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-09T04:34:08.019 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-09T04:34:08.020 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:08.020 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-06-09T04:34:08.124 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T04:34:08.124 INFO:teuthology.orchestra.run.smithi123.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-09T04:34:08.124 INFO:teuthology.orchestra.run.smithi123.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-09T04:34:08.124 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:08.293 DEBUG:teuthology.parallel:result is None 2023-06-09T04:34:08.337 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 1/134 2023-06-09T04:34:08.338 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2/134 2023-06-09T04:34:08.338 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 3/134 2023-06-09T04:34:08.338 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 4/134 2023-06-09T04:34:08.338 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 5/134 2023-06-09T04:34:08.338 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-09T04:34:08.338 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 9/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 10/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 11/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 12/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 13/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 14/134 2023-06-09T04:34:08.339 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 15/134 2023-06-09T04:34:08.340 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 16/134 2023-06-09T04:34:08.340 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 17/134 2023-06-09T04:34:08.340 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 18/134 2023-06-09T04:34:08.340 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 19/134 2023-06-09T04:34:08.340 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 20/134 2023-06-09T04:34:08.340 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 21/134 2023-06-09T04:34:08.340 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 22/134 2023-06-09T04:34:08.341 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 23/134 2023-06-09T04:34:08.341 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 24/134 2023-06-09T04:34:08.347 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 25/134 2023-06-09T04:34:08.348 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 26/134 2023-06-09T04:34:08.348 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 27/134 2023-06-09T04:34:08.348 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 28/134 2023-06-09T04:34:08.348 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 29/134 2023-06-09T04:34:08.348 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 30/134 2023-06-09T04:34:08.348 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 31/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 32/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 33/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 34/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 35/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 36/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 37/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 38/134 2023-06-09T04:34:08.349 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-09T04:34:08.350 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-09T04:34:08.350 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-09T04:34:08.350 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-09T04:34:08.350 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-09T04:34:08.350 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-09T04:34:08.350 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-09T04:34:08.350 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-09T04:34:08.351 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-09T04:34:08.352 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-09T04:34:08.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-09T04:34:08.354 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-09T04:34:08.355 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-09T04:34:08.355 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-09T04:34:08.355 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-09T04:34:08.355 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-09T04:34:08.355 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-09T04:34:08.355 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-09T04:34:08.355 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-09T04:34:08.356 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-09T04:34:08.357 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-09T04:34:08.358 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-09T04:34:08.359 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-09T04:34:08.360 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-09T04:34:08.360 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-09T04:34:08.360 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-09T04:34:08.360 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-09T04:34:08.360 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-09T04:34:08.360 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-09T04:34:08.360 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 129/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 131/134 2023-06-09T04:34:08.361 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T04:34:08.362 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-09T04:34:10.994 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T04:34:10.994 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:01:48 ago on Fri 09 Jun 2023 04:32:22 AM UTC. 2023-06-09T04:34:13.577 INFO:teuthology.orchestra.run.smithi123.stdout:Installed products updated. 2023-06-09T04:34:13.815 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:13.816 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2023-06-09T04:34:13.816 INFO:teuthology.orchestra.run.smithi123.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-09T04:34:13.816 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.816 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.816 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-06-09T04:34:13.816 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.1-7.el8.noarch 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.0.4-10.el8.x86_64 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.817 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.818 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.818 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.818 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.818 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.819 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.820 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.820 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.820 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.820 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.820 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-09T04:34:13.820 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-09T04:34:13.820 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.1-9.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: jq-1.6-3.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.821 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: libpq-13.5-1.el8.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-09T04:34:13.822 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.823 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.823 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-09T04:34:13.823 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.823 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-09T04:34:13.823 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-09T04:34:13.823 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-09T04:34:13.823 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-09T04:34:13.824 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-09T04:34:13.825 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-09T04:34:13.826 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-09T04:34:13.827 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-09T04:34:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-09T04:34:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-09T04:34:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-09T04:34:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-09T04:34:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-09T04:34:13.828 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-09T04:34:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-09T04:34:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-09T04:34:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-09T04:34:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-09T04:34:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-09T04:34:13.829 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-2.6-1.el8.noarch 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.830 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.831 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-09T04:34:13.831 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-09T04:34:13.831 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.831 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-09T04:34:13.831 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-09T04:34:13.831 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-09T04:34:13.831 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-09T04:34:13.832 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-09T04:34:13.833 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-09T04:34:13.834 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-09T04:34:13.834 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-09T04:34:13.834 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:13.834 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-06-09T04:34:13.996 DEBUG:teuthology.parallel:result is None 2023-06-09T04:34:13.996 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-09T04:34:14.224 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-09T04:34:14.251 INFO:teuthology.orchestra.run.smithi092.stdout:18.0.0-4313.ga02638f8.el8 2023-06-09T04:34:14.252 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4313.ga02638f8.el8 2023-06-09T04:34:14.252 INFO:teuthology.task.install:The correct ceph version 18.0.0-4313.ga02638f8 is installed. 2023-06-09T04:34:14.253 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-09T04:34:14.477 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-09T04:34:14.501 INFO:teuthology.orchestra.run.smithi123.stdout:18.0.0-4313.ga02638f8.el8 2023-06-09T04:34:14.501 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4313.ga02638f8.el8 2023-06-09T04:34:14.501 INFO:teuthology.task.install:The correct ceph version 18.0.0-4313.ga02638f8 is installed. 2023-06-09T04:34:14.502 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-09T04:34:14.502 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:14.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-09T04:34:14.548 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:14.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-09T04:34:14.590 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-09T04:34:14.591 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:14.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2023-06-09T04:34:14.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-09T04:34:14.688 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:14.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2023-06-09T04:34:14.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-09T04:34:14.781 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-09T04:34:14.782 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:14.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-09T04:34:14.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-09T04:34:14.875 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:14.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-09T04:34:14.901 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-09T04:34:14.967 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-09T04:34:14.967 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:14.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2023-06-09T04:34:14.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-09T04:34:15.060 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:15.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2023-06-09T04:34:15.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-09T04:34:15.152 INFO:teuthology.run_tasks:Running task ceph... 2023-06-09T04:34:15.219 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-09T04:34:15.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2023-06-09T04:34:15.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2023-06-09T04:34:15.247 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-09T04:34:15.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-09T04:34:15.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-09T04:34:15.313 INFO:tasks.ceph:Creating extra log directories... 2023-06-09T04:34:15.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-09T04:34:15.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-09T04:34:15.384 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-09T04:34:15.385 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-09T04:34:15.385 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-09T04:34:15.385 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:15.386 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-09T04:34:15.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-09T04:34:15.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-09T04:34:15.620 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:15.621 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-09T04:34:15.637 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-09T04:34:15.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-09T04:34:15.801 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-09T04:34:15.802 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-09T04:34:15.802 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7299733', 'branch': 'wip-lxb-fscrypt-20230607-0901', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7299733', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'xiubli-2023-06-09_04:16:27-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-lxb-fscrypt-20230607-0901'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lxb-fscrypt-20230607-0901', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}}, 'owner': 'scheduled_xiubli@teuthology', 'priority': 110, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-lxb-fscrypt-20230607-0901', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi092.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61', 'timestamp': '2023-06-09_04:16:27', 'tube': 'smithi', 'user': 'xiubli', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-09T04:34:15.803 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-09T04:34:15.804 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-09T04:34:15.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2023-06-09T04:34:15.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2023-06-09T04:34:15.918 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:15.919 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2023-06-09T04:34:15.941 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-09T04:34:15.941 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2023-06-09T04:34:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-09T04:34:15.997 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:15.997 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 379099 Links: 1 2023-06-09T04:34:15.997 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:15.998 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:15.998 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-06-09 04:34:02.359468590 +0000 2023-06-09T04:34:15.998 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-06-09 04:31:32.248200568 +0000 2023-06-09T04:34:15.998 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-06-09 04:31:32.248200568 +0000 2023-06-09T04:34:15.998 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-06-09T04:34:15.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-09T04:34:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-06-09T04:34:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-06-09T04:34:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000227136 s, 2.3 MB/s 2023-06-09T04:34:16.062 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-09T04:34:16.119 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2023-06-09T04:34:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-09T04:34:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:16.175 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 378102 Links: 1 2023-06-09T04:34:16.175 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:16.175 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:16.176 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-06-09 04:34:02.359468590 +0000 2023-06-09T04:34:16.176 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-06-09 04:31:32.754185788 +0000 2023-06-09T04:34:16.176 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-06-09 04:31:32.754185788 +0000 2023-06-09T04:34:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-06-09T04:34:16.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-09T04:34:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-06-09T04:34:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-06-09T04:34:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000202548 s, 2.5 MB/s 2023-06-09T04:34:16.240 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-09T04:34:16.297 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2023-06-09T04:34:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-09T04:34:16.353 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:16.353 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 375387 Links: 1 2023-06-09T04:34:16.353 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:16.354 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:16.354 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-06-09 04:34:02.359468590 +0000 2023-06-09T04:34:16.354 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-06-09 04:31:33.270170715 +0000 2023-06-09T04:34:16.354 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-06-09 04:31:33.270170715 +0000 2023-06-09T04:34:16.354 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-06-09T04:34:16.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-09T04:34:16.418 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-06-09T04:34:16.418 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-06-09T04:34:16.418 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000199302 s, 2.6 MB/s 2023-06-09T04:34:16.419 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-09T04:34:16.476 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2023-06-09T04:34:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-09T04:34:16.533 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:16.533 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 375432 Links: 1 2023-06-09T04:34:16.533 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:16.534 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:16.534 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-06-09 04:34:02.359468590 +0000 2023-06-09T04:34:16.534 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-06-09 04:31:33.777155906 +0000 2023-06-09T04:34:16.534 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-06-09 04:31:33.777155906 +0000 2023-06-09T04:34:16.534 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-06-09T04:34:16.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-09T04:34:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-06-09T04:34:16.597 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-06-09T04:34:16.598 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00019185 s, 2.7 MB/s 2023-06-09T04:34:16.598 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-09T04:34:16.656 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:16.656 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:16.656 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2023-06-09T04:34:16.673 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-09T04:34:16.674 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2023-06-09T04:34:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-09T04:34:16.731 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:16.732 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 377214 Links: 1 2023-06-09T04:34:16.732 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:16.732 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:16.732 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-06-09 04:34:08.036874968 +0000 2023-06-09T04:34:16.732 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-06-09 04:31:30.026058943 +0000 2023-06-09T04:34:16.732 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-06-09 04:31:30.026058943 +0000 2023-06-09T04:34:16.732 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-06-09T04:34:16.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-09T04:34:16.797 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-06-09T04:34:16.797 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-06-09T04:34:16.797 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000231236 s, 2.2 MB/s 2023-06-09T04:34:16.798 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-09T04:34:16.855 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2023-06-09T04:34:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-09T04:34:16.911 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 376637 Links: 1 2023-06-09T04:34:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-06-09 04:34:08.036874968 +0000 2023-06-09T04:34:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-06-09 04:31:30.544046325 +0000 2023-06-09T04:34:16.912 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-06-09 04:31:30.544046325 +0000 2023-06-09T04:34:16.912 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-06-09T04:34:16.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-09T04:34:16.975 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-06-09T04:34:16.976 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-06-09T04:34:16.976 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00022296 s, 2.3 MB/s 2023-06-09T04:34:16.977 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-09T04:34:17.034 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2023-06-09T04:34:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-09T04:34:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:17.090 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 373584 Links: 1 2023-06-09T04:34:17.091 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:17.091 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:17.091 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-06-09 04:34:08.036874968 +0000 2023-06-09T04:34:17.091 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-06-09 04:31:31.051033975 +0000 2023-06-09T04:34:17.091 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-06-09 04:31:31.051033975 +0000 2023-06-09T04:34:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-06-09T04:34:17.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-09T04:34:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-06-09T04:34:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-06-09T04:34:17.155 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000252314 s, 2.0 MB/s 2023-06-09T04:34:17.156 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-09T04:34:17.212 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2023-06-09T04:34:17.268 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-09T04:34:17.269 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T04:34:17.269 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 375467 Links: 1 2023-06-09T04:34:17.269 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T04:34:17.269 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T04:34:17.269 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-06-09 04:34:08.036874968 +0000 2023-06-09T04:34:17.269 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-06-09 04:31:31.566021431 +0000 2023-06-09T04:34:17.270 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-06-09 04:31:31.566021431 +0000 2023-06-09T04:34:17.270 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-06-09T04:34:17.270 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-09T04:34:17.334 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-06-09T04:34:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-06-09T04:34:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000195141 s, 2.6 MB/s 2023-06-09T04:34:17.336 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-09T04:34:17.393 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:17.394 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-09T04:34:17.395 INFO:tasks.ceph:Generating config... 2023-06-09T04:34:17.397 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-09T04:34:17.398 INFO:tasks.ceph:[client] debug client = 20 2023-06-09T04:34:17.398 INFO:tasks.ceph:[client] debug ms = 1 2023-06-09T04:34:17.398 INFO:tasks.ceph:[client] pid file = 2023-06-09T04:34:17.399 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-09T04:34:17.399 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-09T04:34:17.400 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-06-09T04:34:17.400 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-09T04:34:17.400 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-06-09T04:34:17.401 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-09T04:34:17.401 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-09T04:34:17.402 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-09T04:34:17.402 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-09T04:34:17.402 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-09T04:34:17.403 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-09T04:34:17.403 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-09T04:34:17.404 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-09T04:34:17.404 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-09T04:34:17.404 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-09T04:34:17.405 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-09T04:34:17.405 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-09T04:34:17.406 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-09T04:34:17.406 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-09T04:34:17.406 INFO:tasks.ceph:[osd] bdev async discard = True 2023-06-09T04:34:17.407 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-06-09T04:34:17.407 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-06-09T04:34:17.408 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-09T04:34:17.408 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-09T04:34:17.408 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-06-09T04:34:17.409 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-06-09T04:34:17.409 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-09T04:34:17.409 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-09T04:34:17.410 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-06-09T04:34:17.410 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-09T04:34:17.411 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-09T04:34:17.411 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-09T04:34:17.411 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-09T04:34:17.412 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-09T04:34:17.412 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-09T04:34:17.413 INFO:tasks.ceph:Setting up mon.a... 2023-06-09T04:34:17.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-09T04:34:17.484 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2023-06-09T04:34:17.487 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-09T04:34:17.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-09T04:34:17.572 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.b', '172.21.15.123'), ('mon.c', '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]')] 2023-06-09T04:34:17.572 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-09T04:34:17.573 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:17.574 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-09T04:34:17.632 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --add b 172.21.15.123 --addv c '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:17.728 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:17.728 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 064bde79-2943-47e5-b941-05f234bd883f 2023-06-09T04:34:17.728 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = pacific 2023-06-09T04:34:17.728 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2023-06-09T04:34:17.729 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 064bde79-2943-47e5-b941-05f234bd883f 2023-06-09T04:34:17.729 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2023-06-09T04:34:17.728261+0000 2023-06-09T04:34:17.729 INFO:teuthology.orchestra.run.smithi092.stdout:created 2023-06-09T04:34:17.728261+0000 2023-06-09T04:34:17.729 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 16 (pacific) 2023-06-09T04:34:17.729 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2023-06-09T04:34:17.729 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2023-06-09T04:34:17.729 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2023-06-09T04:34:17.730 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] mon.c 2023-06-09T04:34:17.730 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-09T04:34:17.731 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-09T04:34:17.750 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 064bde79-2943-47e5-b941-05f234bd883f... 2023-06-09T04:34:17.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-09T04:34:17.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-09T04:34:17.839 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2023-06-09T04:34:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2023-06-09T04:34:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-09T04:34:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2023-06-09T04:34:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2023-06-09T04:34:17.840 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2023-06-09T04:34:17.841 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2023-06-09T04:34:17.842 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2023-06-09T04:34:17.843 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.844 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.92,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on skipped message = False 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 064bde79-2943-47e5-b941-05f234bd883f 2023-06-09T04:34:17.845 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.846 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2023-06-09T04:34:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2023-06-09T04:34:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2023-06-09T04:34:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2023-06-09T04:34:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2023-06-09T04:34:17.846 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.847 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2023-06-09T04:34:17.848 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: bdev async discard = True 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: bdev enable discard = True 2023-06-09T04:34:17.849 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore allocator = bitmap 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 1/20 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 1/20 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 4/10 2023-06-09T04:34:17.850 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2023-06-09T04:34:17.851 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.852 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-06-09T04:34:17.853 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.854 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2023-06-09T04:34:17.855 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-09T04:34:17.856 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2023-06-09T04:34:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-06-09T04:34:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-06-09T04:34:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2023-06-09T04:34:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-06-09T04:34:17.857 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-06-09T04:34:17.857 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2023-06-09T04:34:17.857 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2023-06-09T04:34:17.858 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-06-09T04:34:17.859 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-06-09T04:34:17.860 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2023-06-09T04:34:17.861 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2023-06-09T04:34:17.861 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-09T04:34:17.861 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2023-06-09T04:34:17.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.861 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2023-06-09T04:34:17.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2023-06-09T04:34:17.862 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2023-06-09T04:34:17.863 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.864 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2023-06-09T04:34:17.864 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2023-06-09T04:34:17.864 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2023-06-09T04:34:17.864 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2023-06-09T04:34:17.864 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2023-06-09T04:34:17.864 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2023-06-09T04:34:17.864 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2023-06-09T04:34:17.865 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on skipped message = False 2023-06-09T04:34:17.866 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 064bde79-2943-47e5-b941-05f234bd883f 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2023-06-09T04:34:17.867 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.868 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2023-06-09T04:34:17.869 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2023-06-09T04:34:17.870 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2023-06-09T04:34:17.871 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2023-06-09T04:34:17.872 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.873 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-06-09T04:34:17.874 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2023-06-09T04:34:17.875 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2023-06-09T04:34:17.875 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-06-09T04:34:17.875 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.875 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-09T04:34:17.875 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2023-06-09T04:34:17.875 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2023-06-09T04:34:17.875 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.876 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-09T04:34:17.877 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2023-06-09T04:34:17.878 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2023-06-09T04:34:17.879 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds balancer = 20 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-06-09T04:34:17.880 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-06-09T04:34:17.881 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-09T04:34:17.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-09T04:34:17.966 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-09T04:34:17.966 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:17.966 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-09T04:34:17.983 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:17.983 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-09T04:34:18.040 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:34:18.040 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:18.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-09T04:34:18.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-09T04:34:18.117 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:18.118 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:18.174 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:34:18.175 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:18.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-09T04:34:18.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-09T04:34:18.214 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:18.214 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:18.271 INFO:tasks.ceph:Setting up mon nodes... 2023-06-09T04:34:18.271 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-09T04:34:18.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-09T04:34:18.326 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-09T04:34:18.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-09T04:34:18.387 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-09T04:34:18.390 INFO:tasks.ceph:Setting up mds nodes... 2023-06-09T04:34:18.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-09T04:34:18.445 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-09T04:34:18.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-09T04:34:18.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-09T04:34:18.566 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-09T04:34:18.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-09T04:34:18.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-09T04:34:18.647 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-09T04:34:18.650 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-09T04:34:18.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-09T04:34:18.767 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-09T04:34:18.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-09T04:34:18.795 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-09T04:34:18.795 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-09T04:34:18.838 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-09T04:34:18.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-09T04:34:18.933 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-09T04:34:18.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-09T04:34:18.987 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-09T04:34:19.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-09T04:34:19.083 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-09T04:34:19.095 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-09T04:34:19.096 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-09T04:34:19.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-09T04:34:19.123 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:19.124 INFO:tasks.ceph:role: osd.0 2023-06-09T04:34:19.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:34:19.124 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-09T04:34:19.197 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:19.197 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:19.197 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:19.198 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:19.198 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:19.198 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:19.198 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:19.198 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:19.198 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:19.198 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:19.638 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-06-09T04:34:19.639 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-06-09T04:34:19.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-09T04:34:19.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-09T04:34:19.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-09T04:34:19.927 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:19.928 INFO:tasks.ceph:role: osd.1 2023-06-09T04:34:19.928 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:34:19.928 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-09T04:34:20.001 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:20.002 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:20.002 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:20.002 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:20.003 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:20.003 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:20.003 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:20.422 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-06-09T04:34:20.424 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-06-09T04:34:20.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-09T04:34:20.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-09T04:34:20.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-09T04:34:20.631 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:20.631 INFO:tasks.ceph:role: osd.2 2023-06-09T04:34:20.632 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:34:20.632 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-09T04:34:20.704 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:20.705 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:20.705 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:20.705 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:20.705 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:20.705 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:20.705 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:20.705 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:20.706 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:20.706 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:21.147 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-06-09T04:34:21.148 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-06-09T04:34:21.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-09T04:34:21.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-09T04:34:21.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-09T04:34:21.423 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:21.423 INFO:tasks.ceph:role: osd.3 2023-06-09T04:34:21.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:34:21.424 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-09T04:34:21.497 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:21.497 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:21.497 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:21.497 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:21.497 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:21.497 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:21.498 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:21.498 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:21.498 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:21.498 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:21.930 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-06-09T04:34:21.931 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-06-09T04:34:21.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-09T04:34:22.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-09T04:34:22.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:22.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:22.250+0000 7f5896df9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-09T04:34:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:22.250+0000 7f5896df9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-09T04:34:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:22.250+0000 7f5896df9640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-09T04:34:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:22.250+0000 7f5896df9640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-09T04:34:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:22.250+0000 7f5896df9640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-09T04:34:22.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:22.250+0000 7f5896df9640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-09T04:34:25.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-09T04:34:25.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:25.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:25.242+0000 7fe08d497640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-09T04:34:25.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:25.242+0000 7fe08d497640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-09T04:34:25.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:25.242+0000 7fe08d497640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-09T04:34:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:25.242+0000 7fe08d497640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-09T04:34:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:25.242+0000 7fe08d497640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-09T04:34:25.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:25.243+0000 7fe08d497640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-09T04:34:28.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-09T04:34:28.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:28.225+0000 7ff3a22d0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-09T04:34:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:28.225+0000 7ff3a22d0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-09T04:34:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:28.225+0000 7ff3a22d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-09T04:34:28.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:28.225+0000 7ff3a22d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-09T04:34:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:28.225+0000 7ff3a22d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-09T04:34:28.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:28.226+0000 7ff3a22d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-09T04:34:31.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-09T04:34:31.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:31.219+0000 7f23bca45640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-09T04:34:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:31.219+0000 7f23bca45640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-09T04:34:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:31.219+0000 7f23bca45640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-09T04:34:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:31.219+0000 7f23bca45640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-09T04:34:31.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:31.219+0000 7f23bca45640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-09T04:34:31.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:31.219+0000 7f23bca45640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-09T04:34:34.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-09T04:34:34.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-09T04:34:34.147 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:34.147 INFO:tasks.ceph:role: osd.4 2023-06-09T04:34:34.148 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:34:34.148 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-09T04:34:34.226 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:34.226 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:34.226 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:34.227 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:34.227 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:34.227 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:34.227 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:34.227 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:34.227 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:34.227 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:34.660 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-06-09T04:34:34.663 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-06-09T04:34:34.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-09T04:34:34.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-09T04:34:34.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-09T04:34:34.957 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:34.958 INFO:tasks.ceph:role: osd.5 2023-06-09T04:34:34.958 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:34:34.958 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-09T04:34:35.038 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:35.038 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:35.038 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:35.039 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:35.039 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:35.039 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:35.039 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:35.039 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:35.039 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:35.039 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:35.479 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-06-09T04:34:35.480 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-06-09T04:34:35.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-09T04:34:35.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-09T04:34:35.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-09T04:34:35.721 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:35.721 INFO:tasks.ceph:role: osd.6 2023-06-09T04:34:35.721 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:34:35.721 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-09T04:34:35.796 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:35.797 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:35.797 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:35.797 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:35.797 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:35.797 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:35.797 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:35.798 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:35.798 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:35.798 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:36.249 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-06-09T04:34:36.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-06-09T04:34:36.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-09T04:34:36.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-09T04:34:36.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-09T04:34:36.526 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T04:34:36.526 INFO:tasks.ceph:role: osd.7 2023-06-09T04:34:36.526 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:34:36.527 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-09T04:34:36.600 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T04:34:36.600 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T04:34:36.600 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T04:34:36.600 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T04:34:36.601 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T04:34:36.601 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-06-09T04:34:36.601 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T04:34:36.601 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T04:34:36.601 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T04:34:36.601 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T04:34:37.031 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-06-09T04:34:37.033 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-06-09T04:34:37.034 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-09T04:34:37.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-09T04:34:37.226 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:37.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:37.346+0000 7f133360c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-09T04:34:37.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:37.346+0000 7f133360c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-09T04:34:37.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:37.346+0000 7f133360c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-09T04:34:37.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:37.346+0000 7f133360c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-09T04:34:37.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:37.346+0000 7f133360c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-09T04:34:37.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:37.346+0000 7f133360c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-09T04:34:40.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-09T04:34:40.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:40.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:40.346+0000 7f5f72939640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:40.347+0000 7f5f72939640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:40.347+0000 7f5f72939640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:40.347+0000 7f5f72939640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:40.347+0000 7f5f72939640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-09T04:34:40.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:40.347+0000 7f5f72939640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-09T04:34:43.213 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-09T04:34:43.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:43.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:43.340+0000 7f308d486640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-09T04:34:43.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:43.340+0000 7f308d486640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-09T04:34:43.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:43.340+0000 7f308d486640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-09T04:34:43.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:43.340+0000 7f308d486640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-09T04:34:43.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:43.340+0000 7f308d486640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-09T04:34:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:43.341+0000 7f308d486640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-09T04:34:45.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:34:45.806 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:34:46.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-09T04:34:46.240 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:46.335+0000 7f50dc554640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-09T04:34:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:46.335+0000 7f50dc554640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-09T04:34:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:46.335+0000 7f50dc554640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-09T04:34:46.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:46.335+0000 7f50dc554640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-09T04:34:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:46.335+0000 7f50dc554640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-09T04:34:46.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:46.336+0000 7f50dc554640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-09T04:34:49.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-09T04:34:49.233 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-09T04:34:49.233 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:49.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-09T04:34:49.265 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:49.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-09T04:34:49.336 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:49.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-09T04:34:49.409 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:49.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-09T04:34:49.481 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:49.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-09T04:34:49.552 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:49.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-09T04:34:49.624 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:49.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-09T04:34:49.696 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:49.696 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-09T04:34:49.728 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:49.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-09T04:34:49.799 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:49.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-09T04:34:49.871 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:49.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-09T04:34:49.944 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:49.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-09T04:34:50.015 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:50.015 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-09T04:34:50.086 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:50.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-09T04:34:50.156 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:50.156 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-09T04:34:50.176 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:50.177 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-09T04:34:50.235 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:50.235 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-09T04:34:50.254 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:50.254 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-09T04:34:50.312 INFO:tasks.ceph:Adding keys to all mons... 2023-06-09T04:34:50.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-09T04:34:50.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-09T04:34:50.355 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2023-06-09T04:34:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4JkVsx/ExAA2ic1kHrajB36QXO1QRmwqg== 2023-06-09T04:34:50.355 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-06-09T04:34:50.355 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4JkkRWHGhAANRHu29QVa0w4jNy4uoHuxQ== 2023-06-09T04:34:50.355 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-06-09T04:34:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4Jkipe+IRAAg2IxPNEVdY1SZWDCIvW7pg== 2023-06-09T04:34:50.356 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2023-06-09T04:34:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDOq4JkBgz9DhAAtsLR0VX+r0N4C8xns2mSUQ== 2023-06-09T04:34:50.356 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2023-06-09T04:34:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDRq4JkqGeCDhAAo38JWWjdNexDgVe4IlyN6A== 2023-06-09T04:34:50.356 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2023-06-09T04:34:50.356 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDUq4JkC1+GDRAAFnmFCsWYqNeqNxVNcRxw+g== 2023-06-09T04:34:50.357 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2023-06-09T04:34:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDXq4Jk85MhDRAA6v4qBxXw6YMskDJ/9JJJew== 2023-06-09T04:34:50.357 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2023-06-09T04:34:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4JkAq4gFxAAumlBS5Z3WMHfT6PFQ1VgxA== 2023-06-09T04:34:50.357 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-06-09T04:34:50.357 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4Jkq/ObJhAAwcBPqWUytLIzHn5o7SxDFg== 2023-06-09T04:34:50.357 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4Jkyqi8LRAA+ZzBtsM0fkmU23EvTDskBA== 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDdq4Jk/oi0FBAAbtRF0o8GPx+kc+MH4qvbTg== 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDgq4JkBnGxFBAAniJ+fC2f163u0u5IlAk6tw== 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDjq4JkOllOFBAAuxY5IpMVak8cpjCU1RGQKQ== 2023-06-09T04:34:50.358 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDmq4JkmTMTFBAA/eQl2z5o21anTixCLtyZFQ== 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4JkUoD5MRAAx3w8zlcMnuTeKsysCg+Oxg== 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4JkoBCjNxAAwrikYjaSgWFfBT2TsAafVQ== 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDKq4JkPankOhAAQqfsTOny57jlvaUiBDn1Kg== 2023-06-09T04:34:50.359 INFO:teuthology.orchestra.run.smithi092.stdout:[client.3] 2023-06-09T04:34:50.360 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDLq4Jkrif6BBAAdoy3vg6SwBtDJjziNCZxiw== 2023-06-09T04:34:50.379 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2023-06-09T04:34:50.380 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4JkVsx/ExAA2ic1kHrajB36QXO1QRmwqg== 2023-06-09T04:34:50.380 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-06-09T04:34:50.380 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4JkkRWHGhAANRHu29QVa0w4jNy4uoHuxQ== 2023-06-09T04:34:50.380 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-06-09T04:34:50.380 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4Jkipe+IRAAg2IxPNEVdY1SZWDCIvW7pg== 2023-06-09T04:34:50.380 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDOq4JkBgz9DhAAtsLR0VX+r0N4C8xns2mSUQ== 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDRq4JkqGeCDhAAo38JWWjdNexDgVe4IlyN6A== 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDUq4JkC1+GDRAAFnmFCsWYqNeqNxVNcRxw+g== 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDXq4Jk85MhDRAA6v4qBxXw6YMskDJ/9JJJew== 2023-06-09T04:34:50.381 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2023-06-09T04:34:50.382 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4JkAq4gFxAAumlBS5Z3WMHfT6PFQ1VgxA== 2023-06-09T04:34:50.382 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-06-09T04:34:50.382 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4Jkq/ObJhAAwcBPqWUytLIzHn5o7SxDFg== 2023-06-09T04:34:50.382 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-06-09T04:34:50.382 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4Jkyqi8LRAA+ZzBtsM0fkmU23EvTDskBA== 2023-06-09T04:34:50.382 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2023-06-09T04:34:50.382 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDdq4Jk/oi0FBAAbtRF0o8GPx+kc+MH4qvbTg== 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDgq4JkBnGxFBAAniJ+fC2f163u0u5IlAk6tw== 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDjq4JkOllOFBAAuxY5IpMVak8cpjCU1RGQKQ== 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDmq4JkmTMTFBAA/eQl2z5o21anTixCLtyZFQ== 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2023-06-09T04:34:50.383 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4JkUoD5MRAAx3w8zlcMnuTeKsysCg+Oxg== 2023-06-09T04:34:50.384 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2023-06-09T04:34:50.384 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4JkoBCjNxAAwrikYjaSgWFfBT2TsAafVQ== 2023-06-09T04:34:50.384 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2023-06-09T04:34:50.384 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDKq4JkPankOhAAQqfsTOny57jlvaUiBDn1Kg== 2023-06-09T04:34:50.384 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2023-06-09T04:34:50.384 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDLq4Jkrif6BBAAdoy3vg6SwBtDJjziNCZxiw== 2023-06-09T04:34:50.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T04:34:50.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T04:34:50.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.541 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.759 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T04:34:50.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T04:34:50.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.867 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.924 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T04:34:50.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:50.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:51.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:51.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:51.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:51.090 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:51.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:51.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T04:34:51.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T04:34:51.426 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-09T04:34:51.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-09T04:34:51.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-09T04:34:51.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-09T04:34:51.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-09T04:34:51.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-09T04:34:51.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-09T04:34:51.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-09T04:34:51.789 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-09T04:34:51.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-09T04:34:51.911 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:51.913 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-09T04:34:51.967 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-09T04:34:51.967 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-09T04:34:51.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-09T04:34:51.971 INFO:tasks.ceph.mon.a:Started 2023-06-09T04:34:51.971 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-09T04:34:51.971 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-09T04:34:52.009 INFO:tasks.ceph.mon.b:Started 2023-06-09T04:34:52.009 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-09T04:34:52.010 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-09T04:34:52.011 INFO:tasks.ceph.mon.c:Started 2023-06-09T04:34:52.011 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-09T04:34:52.012 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-09T04:34:52.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-09T04:34:52.014 INFO:tasks.ceph.mgr.y:Started 2023-06-09T04:34:52.014 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-09T04:34:52.015 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-09T04:34:52.016 INFO:tasks.ceph.mgr.x:Started 2023-06-09T04:34:52.016 INFO:tasks.ceph:Setting crush tunables to default 2023-06-09T04:34:52.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2023-06-09T04:34:52.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.137+0000 7f17c4c54700 1 Processor -- start 2023-06-09T04:34:52.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.137+0000 7f17c4c54700 1 -- start start 2023-06-09T04:34:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.138+0000 7f17c4c54700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 0x7f17c00713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.138+0000 7f17c4c54700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17c015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.138+0000 7f17c4c54700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c00ce190 con 0x7f17c00ce3d0 2023-06-09T04:34:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.138+0000 7f17c4c54700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c0072350 con 0x7f17c00aed50 2023-06-09T04:34:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.138+0000 7f17c4c54700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0071ab0 con 0x7f17c015d470 2023-06-09T04:34:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bdd9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 0x7f17c00713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bed9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17c015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bdd9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 0x7f17c00713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:57202/0 (socket says 172.21.15.92:57202) 2023-06-09T04:34:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bed9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17c015d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:45706/0 (socket says 172.21.15.92:45706) 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17be59c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f17c00ce3d0 0x7f17c00ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:57620/0 (socket says 172.21.15.92:57620) 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bdd9b700 1 -- 172.21.15.92:0/77204230 learned_addr learned my addr 172.21.15.92:0/77204230 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bed9d700 1 -- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 msgr2=0x7f17c015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bdd9b700 1 -- 172.21.15.92:0/77204230 >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 msgr2=0x7f17c00713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bed9d700 1 -- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 msgr2=0x7f17c015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bdd9b700 1 -- 172.21.15.92:0/77204230 >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 msgr2=0x7f17c00713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bed9d700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17c015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T04:34:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bdd9b700 1 --2- 172.21.15.92:0/77204230 >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 0x7f17c00713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T04:34:52.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bdd9b700 1 --2- 172.21.15.92:0/77204230 >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 0x7f17c00713b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T04:34:52.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.139+0000 7f17bed9d700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17c015d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T04:34:52.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.321+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330773632 0 0) 0x7f17c00ce190 con 0x7f17c00ce3d0 2023-06-09T04:34:52.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.321+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17a80035b0 con 0x7f17c00ce3d0 2023-06-09T04:34:52.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3139370158 0 0) 0x7f17a80035b0 con 0x7f17c00ce3d0 2023-06-09T04:34:52.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 msgr2=0x7f17c015d8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:34:52.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17c015d8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 msgr2=0x7f17c00713b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:34:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 --2- 172.21.15.92:0/77204230 >> v2:172.21.15.123:3300/0 conn(0x7f17c00aed50 0x7f17c00713b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0071d90 con 0x7f17c00ce3d0 2023-06-09T04:34:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f17b4001670 con 0x7f17c00ce3d0 2023-06-09T04:34:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 >> v1:172.21.15.123:6789/0 conn(0x7f17c00ce3d0 legacy=0x7f17c00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17a8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17a8004d90 0x7f17a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0072350 con 0x7f17a8004030 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0071ab0 con 0x7f17a8004d90 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a800bb10 con 0x7f17c015d470 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bdd9b700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17be59c700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17a8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bed9d700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17a8004d90 0x7f17a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bdd9b700 1 -- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 msgr2=0x7f17a8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bdd9b700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c015d470 0x7f17a8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bdd9b700 1 -- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17a8004d90 msgr2=0x7f17a8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bdd9b700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17a8004d90 0x7f17a8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.326+0000 7f17bdd9b700 1 -- 172.21.15.92:0/77204230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a80035b0 con 0x7f17a8004030 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.327+0000 7f17bdd9b700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17a8004850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17b000bdc0 tx=0x7f17a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71082097d53cfed server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.327+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000b860 con 0x7f17a8004030 2023-06-09T04:34:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.327+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b0010d70 con 0x7f17a8004030 2023-06-09T04:34:52.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.327+0000 7f17bd59a700 1 -- 172.21.15.92:0/77204230 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0010ee0 con 0x7f17a8004030 2023-06-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.327+0000 7f17c4c54700 1 -- 172.21.15.92:0/77204230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 msgr2=0x7f17a8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.327+0000 7f17c4c54700 1 --2- 172.21.15.92:0/77204230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17a8004850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17b000bdc0 tx=0x7f17a8005880 comp rx=0 tx=0).stop 2023-06-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.327+0000 7f17bed9d700 1 -- 172.21.15.92:0/77204230 reap_dead start 2023-06-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.328+0000 7f17c4c54700 1 -- 172.21.15.92:0/77204230 shutdown_connections 2023-06-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.328+0000 7f17c4c54700 1 -- 172.21.15.92:0/77204230 >> 172.21.15.92:0/77204230 conn(0x7f17c00cc380 msgr2=0x7f17c00cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.328+0000 7f17c4c54700 1 -- 172.21.15.92:0/77204230 shutdown_connections 2023-06-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.328+0000 7f17c4c54700 1 -- 172.21.15.92:0/77204230 wait complete. 2023-06-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.329+0000 7f17c4c54700 1 Processor -- start 2023-06-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.329+0000 7f17c4c54700 1 -- start start 2023-06-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17c4c54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17c00c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17c4c54700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17c015b8e0 0x7f17c015c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17c4c54700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c0153ee0 0x7f17c00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17c4c54700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c015b4c0 con 0x7f17a8004030 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17c4c54700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c015cbb0 con 0x7f17c015b8e0 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17c4c54700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c015ab20 con 0x7f17c0153ee0 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17be59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17c00c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17bdd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17c015b8e0 0x7f17c015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17be59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17c00c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36824/0 (socket says 172.21.15.92:36824) 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17bed9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c0153ee0 0x7f17c00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.330+0000 7f17be59c700 1 -- 172.21.15.92:0/705156604 learned_addr learned my addr 172.21.15.92:0/705156604 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17be59c700 1 -- 172.21.15.92:0/705156604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c0153ee0 msgr2=0x7f17c00b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17be59c700 1 --2- 172.21.15.92:0/705156604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c0153ee0 0x7f17c00b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17be59c700 1 -- 172.21.15.92:0/705156604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17c015b8e0 msgr2=0x7f17c015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17be59c700 1 --2- 172.21.15.92:0/705156604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17c015b8e0 0x7f17c015c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17be59c700 1 -- 172.21.15.92:0/705156604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c015ae00 con 0x7f17a8004030 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17be59c700 1 --2- 172.21.15.92:0/705156604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17c00c5c30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000fb0 tx=0x7f17b40075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17aeffd700 1 -- 172.21.15.92:0/705156604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400f690 con 0x7f17a8004030 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17aeffd700 1 -- 172.21.15.92:0/705156604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b400f810 con 0x7f17a8004030 2023-06-09T04:34:52.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c015b0e0 con 0x7f17a8004030 2023-06-09T04:34:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17aeffd700 1 -- 172.21.15.92:0/705156604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400fbc0 con 0x7f17a8004030 2023-06-09T04:34:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c00b08f0 con 0x7f17a8004030 2023-06-09T04:34:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17aeffd700 1 -- 172.21.15.92:0/705156604 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f17b401d460 con 0x7f17a8004030 2023-06-09T04:34:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.331+0000 7f17aeffd700 1 -- 172.21.15.92:0/705156604 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f17b4013500 con 0x7f17a8004030 2023-06-09T04:34:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.332+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c004e8e0 con 0x7f17a8004030 2023-06-09T04:34:52.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.336+0000 7f17aeffd700 1 -- 172.21.15.92:0/705156604 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f17b401d900 con 0x7f17a8004030 2023-06-09T04:34:52.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.412+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f17c006a8e0 con 0x7f17a8004030 2023-06-09T04:34:52.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.415+0000 7f17aeffd700 1 -- 172.21.15.92:0/705156604 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f17b401daa0 con 0x7f17a8004030 2023-06-09T04:34:52.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 msgr2=0x7f17c00c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 --2- 172.21.15.92:0/705156604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17c00c5c30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000fb0 tx=0x7f17b40075f0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 shutdown_connections 2023-06-09T04:34:52.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 --2- 172.21.15.92:0/705156604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f17c0153ee0 0x7f17c00b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 --2- 172.21.15.92:0/705156604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17a8004030 0x7f17c00c5c30 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000fb0 tx=0x7f17b40075f0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 --2- 172.21.15.92:0/705156604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17c015b8e0 0x7f17c015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 >> 172.21.15.92:0/705156604 conn(0x7f17c00cc380 msgr2=0x7f17c00cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 shutdown_connections 2023-06-09T04:34:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:52.416+0000 7f17c4c54700 1 -- 172.21.15.92:0/705156604 wait complete. 2023-06-09T04:34:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2023-06-09T04:34:52.428 INFO:tasks.ceph:check_enable_crimson: False 2023-06-09T04:34:52.429 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-09T04:34:52.429 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:52.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-09T04:34:52.438 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:52.437+0000 7f01cd798200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-09T04:34:52.449 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:52.449+0000 7f0349711200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-09T04:34:52.455 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:52.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-09T04:34:52.503 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:52.502+0000 7f01cd798200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-09T04:34:52.520 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:52.519+0000 7f0349711200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-09T04:34:52.521 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:52.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-09T04:34:52.578 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:52.577+0000 7f01cd798200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-09T04:34:52.589 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:34:52.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-09T04:34:52.601 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:52.600+0000 7f0349711200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-09T04:34:52.626 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:52.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-09T04:34:52.652 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:52.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-09T04:34:52.658 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:52.657+0000 7f01cd798200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-09T04:34:52.683 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:52.683 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-09T04:34:52.684 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:52.683+0000 7f0349711200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-09T04:34:52.749 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:34:52.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-09T04:34:52.807 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:52.806+0000 7f01cd798200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-09T04:34:52.814 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 0c9fd229-6037-49a3-a19d-2e9b9f077b05 0 2023-06-09T04:34:52.826 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:52.826+0000 7f0349711200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-09T04:34:52.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.972+0000 7fa6c034d700 1 Processor -- start 2023-06-09T04:34:52.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.972+0000 7fa6c034d700 1 -- start start 2023-06-09T04:34:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6c034d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b806a4c0 0x7fa6b80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6c034d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b8071dc0 con 0x7fa6b80aee60 2023-06-09T04:34:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6c034d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b80cf030 con 0x7fa6b80aea90 2023-06-09T04:34:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6c034d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b815edf0 con 0x7fa6b806a4c0 2023-06-09T04:34:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6be0e9700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa6b80aea90 0x7fa6b8069db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:50156/0 (socket says 172.21.15.123:50156) 2023-06-09T04:34:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6be0e9700 1 -- 172.21.15.123:0/2490795980 learned_addr learned my addr 172.21.15.123:0/2490795980 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-06-09T04:34:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6be8ea700 1 --2- 172.21.15.123:0/2490795980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b806a4c0 0x7fa6b80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6bd0e7700 1 -- 172.21.15.123:0/2490795980 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601579140 0 0) 0x7fa6b80cf030 con 0x7fa6b80aea90 2023-06-09T04:34:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6bd0e7700 1 -- 172.21.15.123:0/2490795980 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a40035b0 con 0x7fa6b80aea90 2023-06-09T04:34:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6be8ea700 1 -- 172.21.15.123:0/2490795980 >> v1:172.21.15.123:6789/0 conn(0x7fa6b80aea90 legacy=0x7fa6b8069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6be8ea700 1 -- 172.21.15.123:0/2490795980 >> v1:172.21.15.92:6789/0 conn(0x7fa6b80aee60 legacy=0x7fa6b806ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6be8ea700 1 -- 172.21.15.123:0/2490795980 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80cf2d0 con 0x7fa6b806a4c0 2023-06-09T04:34:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6be8ea700 1 --2- 172.21.15.123:0/2490795980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b806a4c0 0x7fa6b80cce60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009dc0 tx=0x7fa6b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd783208865c439e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:52.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.973+0000 7fa6bd0e7700 1 -- 172.21.15.123:0/2490795980 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000e070 con 0x7fa6b806a4c0 2023-06-09T04:34:52.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6bd0e7700 1 -- 172.21.15.123:0/2490795980 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b000bb20 con 0x7fa6b806a4c0 2023-06-09T04:34:52.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6bd0e7700 1 -- 172.21.15.123:0/2490795980 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000be90 con 0x7fa6b806a4c0 2023-06-09T04:34:52.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6c034d700 1 -- 172.21.15.123:0/2490795980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b806a4c0 msgr2=0x7fa6b80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6c034d700 1 --2- 172.21.15.123:0/2490795980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b806a4c0 0x7fa6b80cce60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009dc0 tx=0x7fa6b000b330 comp rx=0 tx=0).stop 2023-06-09T04:34:52.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6c034d700 1 -- 172.21.15.123:0/2490795980 shutdown_connections 2023-06-09T04:34:52.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6c034d700 1 --2- 172.21.15.123:0/2490795980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b806a4c0 0x7fa6b80cce60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6c034d700 1 -- 172.21.15.123:0/2490795980 >> 172.21.15.123:0/2490795980 conn(0x7fa6b80cc380 msgr2=0x7fa6b80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:52.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6c034d700 1 -- 172.21.15.123:0/2490795980 shutdown_connections 2023-06-09T04:34:52.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.974+0000 7fa6c034d700 1 -- 172.21.15.123:0/2490795980 wait complete. 2023-06-09T04:34:52.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.975+0000 7fa6c034d700 1 Processor -- start 2023-06-09T04:34:52.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.975+0000 7fa6c034d700 1 -- start start 2023-06-09T04:34:52.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6c034d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6b806a4c0 0x7fa6b80cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6c034d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b80aea90 0x7fa6b8151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6c034d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8151f40 0x7fa6b80ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6c034d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80c9f50 con 0x7fa6b806a4c0 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6c034d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81523a0 con 0x7fa6b8151f40 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6c034d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80caf60 con 0x7fa6b80aea90 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b80aea90 0x7fa6b8151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b80aea90 0x7fa6b8151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44522/0 (socket says 172.21.15.123:44522) 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 -- 172.21.15.123:0/1028178441 learned_addr learned my addr 172.21.15.123:0/1028178441 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6be8ea700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8151f40 0x7fa6b80ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6be0e9700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6b806a4c0 0x7fa6b80cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:52.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 -- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8151f40 msgr2=0x7fa6b80ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8151f40 0x7fa6b80ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 -- 172.21.15.123:0/1028178441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6b806a4c0 msgr2=0x7fa6b80cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:52.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6b806a4c0 0x7fa6b80cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:52.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.976+0000 7fa6bd8e8700 1 -- 172.21.15.123:0/1028178441 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b0009a70 con 0x7fa6b80aea90 2023-06-09T04:34:52.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6bd8e8700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b80aea90 0x7fa6b8151a00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800c500 tx=0x7fa6a800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:52.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6aeffd700 1 -- 172.21.15.123:0/1028178441 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80052d0 con 0x7fa6b80aea90 2023-06-09T04:34:52.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6aeffd700 1 -- 172.21.15.123:0/1028178441 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a8005430 con 0x7fa6b80aea90 2023-06-09T04:34:52.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6aeffd700 1 -- 172.21.15.123:0/1028178441 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8006aa0 con 0x7fa6b80aea90 2023-06-09T04:34:52.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b80ca1a0 con 0x7fa6b80aea90 2023-06-09T04:34:52.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b80ca760 con 0x7fa6b80aea90 2023-06-09T04:34:52.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6aeffd700 1 -- 172.21.15.123:0/1028178441 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa6a8003980 con 0x7fa6b80aea90 2023-06-09T04:34:52.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.977+0000 7fa6aeffd700 1 -- 172.21.15.123:0/1028178441 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa6a80119e0 con 0x7fa6b80aea90 2023-06-09T04:34:52.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.978+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b804e8e0 con 0x7fa6b80aea90 2023-06-09T04:34:52.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:52.980+0000 7fa6aeffd700 1 -- 172.21.15.123:0/1028178441 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa6a8020060 con 0x7fa6b80aea90 2023-06-09T04:34:53.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.062+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0c9fd229-6037-49a3-a19d-2e9b9f077b05", "id": 0} v 0) v1 -- 0x7fa6b806a9b0 con 0x7fa6b80aea90 2023-06-09T04:34:53.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.067+0000 7fa6aeffd700 1 -- 172.21.15.123:0/1028178441 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c9fd229-6037-49a3-a19d-2e9b9f077b05", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa6a8003d70 con 0x7fa6b80aea90 2023-06-09T04:34:53.069 INFO:teuthology.orchestra.run.smithi123.stdout:0 2023-06-09T04:34:53.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b80aea90 msgr2=0x7fa6b8151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b80aea90 0x7fa6b8151a00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800c500 tx=0x7fa6a800c810 comp rx=0 tx=0).stop 2023-06-09T04:34:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 shutdown_connections 2023-06-09T04:34:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6b8151f40 0x7fa6b80ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6b806a4c0 0x7fa6b80cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 --2- 172.21.15.123:0/1028178441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa6b80aea90 0x7fa6b8151a00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 >> 172.21.15.123:0/1028178441 conn(0x7fa6b80cc380 msgr2=0x7fa6b80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 shutdown_connections 2023-06-09T04:34:53.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.069+0000 7fa6c034d700 1 -- 172.21.15.123:0/1028178441 wait complete. 2023-06-09T04:34:53.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 9517b22e-f8d8-469d-9805-b7d3018f29bc 1 2023-06-09T04:34:53.151 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:53.151+0000 7f0349711200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-09T04:34:53.178 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:53.177+0000 7f01cd798200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-09T04:34:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.203+0000 7fd935bb4700 1 Processor -- start 2023-06-09T04:34:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.203+0000 7fd935bb4700 1 -- start start 2023-06-09T04:34:53.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd935bb4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9300aed50 0x7fd9300713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd935bb4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd93015d470 0x7fd93015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd935bb4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9300ce190 con 0x7fd9300ce3d0 2023-06-09T04:34:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd935bb4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd930072350 con 0x7fd9300aed50 2023-06-09T04:34:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd935bb4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd930071ab0 con 0x7fd93015d470 2023-06-09T04:34:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd93015d470 0x7fd93015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd93015d470 0x7fd93015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40838/0 (socket says 172.21.15.123:40838) 2023-06-09T04:34:53.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 -- 172.21.15.123:0/4201789837 learned_addr learned my addr 172.21.15.123:0/4201789837 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92effd700 1 --2- 172.21.15.123:0/4201789837 >> v2:172.21.15.92:3300/0 conn(0x7fd9300aed50 0x7fd9300713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 -- 172.21.15.123:0/4201789837 >> v2:172.21.15.92:3300/0 conn(0x7fd9300aed50 msgr2=0x7fd9300713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 --2- 172.21.15.123:0/4201789837 >> v2:172.21.15.92:3300/0 conn(0x7fd9300aed50 0x7fd9300713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 -- 172.21.15.123:0/4201789837 >> v1:172.21.15.92:6789/0 conn(0x7fd9300ce3d0 legacy=0x7fd9300ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 -- 172.21.15.123:0/4201789837 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930071d90 con 0x7fd93015d470 2023-06-09T04:34:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92ffff700 1 --2- 172.21.15.123:0/4201789837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93015d470 0x7fd93015d8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd930070920 tx=0x7fd92800d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ccb2b1669c8afd8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92e7fc700 1 -- 172.21.15.123:0/4201789837 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928010070 con 0x7fd93015d470 2023-06-09T04:34:53.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92e7fc700 1 -- 172.21.15.123:0/4201789837 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92800dce0 con 0x7fd93015d470 2023-06-09T04:34:53.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.204+0000 7fd92e7fc700 1 -- 172.21.15.123:0/4201789837 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800b310 con 0x7fd93015d470 2023-06-09T04:34:53.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 -- 172.21.15.123:0/4201789837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93015d470 msgr2=0x7fd93015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 --2- 172.21.15.123:0/4201789837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93015d470 0x7fd93015d8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd930070920 tx=0x7fd92800d430 comp rx=0 tx=0).stop 2023-06-09T04:34:53.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 -- 172.21.15.123:0/4201789837 shutdown_connections 2023-06-09T04:34:53.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 --2- 172.21.15.123:0/4201789837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd93015d470 0x7fd93015d8b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 --2- 172.21.15.123:0/4201789837 >> v2:172.21.15.92:3300/0 conn(0x7fd9300aed50 0x7fd9300713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 -- 172.21.15.123:0/4201789837 >> 172.21.15.123:0/4201789837 conn(0x7fd9300cc380 msgr2=0x7fd9300cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 -- 172.21.15.123:0/4201789837 shutdown_connections 2023-06-09T04:34:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.205+0000 7fd935bb4700 1 -- 172.21.15.123:0/4201789837 wait complete. 2023-06-09T04:34:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.206+0000 7fd935bb4700 1 Processor -- start 2023-06-09T04:34:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.206+0000 7fd935bb4700 1 -- start start 2023-06-09T04:34:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd935bb4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9300aed50 0x7fd93014c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd935bb4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9300ce3d0 0x7fd93015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd935bb4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd93015d470 0x7fd930157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd935bb4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9301f27b0 con 0x7fd9300aed50 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd935bb4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300ac480 con 0x7fd9300ce3d0 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd935bb4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300ac760 con 0x7fd93015d470 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd93015d470 0x7fd930157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9300ce3d0 0x7fd93015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd93015d470 0x7fd930157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44536/0 (socket says 172.21.15.123:44536) 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 -- 172.21.15.123:0/677284526 learned_addr learned my addr 172.21.15.123:0/677284526 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:53.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92f7fe700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9300aed50 0x7fd93014c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 -- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9300ce3d0 msgr2=0x7fd93015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9300ce3d0 0x7fd93015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 -- 172.21.15.123:0/677284526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9300aed50 msgr2=0x7fd93014c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9300aed50 0x7fd93014c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92ffff700 1 -- 172.21.15.123:0/677284526 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92800bd40 con 0x7fd93015d470 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.207+0000 7fd92effd700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9300ce3d0 0x7fd93015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd92ffff700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd93015d470 0x7fd930157f70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d780 tx=0x7fd928004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd917fff700 1 -- 172.21.15.123:0/677284526 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928009a10 con 0x7fd93015d470 2023-06-09T04:34:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd917fff700 1 -- 172.21.15.123:0/677284526 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd928009b70 con 0x7fd93015d470 2023-06-09T04:34:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9300aca40 con 0x7fd93015d470 2023-06-09T04:34:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd917fff700 1 -- 172.21.15.123:0/677284526 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800b4c0 con 0x7fd93015d470 2023-06-09T04:34:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9300ad000 con 0x7fd93015d470 2023-06-09T04:34:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd917fff700 1 -- 172.21.15.123:0/677284526 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd928004340 con 0x7fd93015d470 2023-06-09T04:34:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.208+0000 7fd917fff700 1 -- 172.21.15.123:0/677284526 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fd92801db80 con 0x7fd93015d470 2023-06-09T04:34:53.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.209+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93006be50 con 0x7fd93015d470 2023-06-09T04:34:53.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.212+0000 7fd917fff700 1 -- 172.21.15.123:0/677284526 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fd928021020 con 0x7fd93015d470 2023-06-09T04:34:53.225 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:53.225+0000 7f0349711200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-09T04:34:53.258 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:53.257+0000 7f01cd798200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-09T04:34:53.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.296+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9517b22e-f8d8-469d-9805-b7d3018f29bc", "id": 1} v 0) v1 -- 0x7fd93004e8e0 con 0x7fd93015d470 2023-06-09T04:34:53.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.304+0000 7fd917fff700 1 -- 172.21.15.123:0/677284526 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9517b22e-f8d8-469d-9805-b7d3018f29bc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd92801dd90 con 0x7fd93015d470 2023-06-09T04:34:53.305 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-06-09T04:34:53.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd93015d470 msgr2=0x7fd930157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd93015d470 0x7fd930157f70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d780 tx=0x7fd928004ad0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 shutdown_connections 2023-06-09T04:34:53.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd93015d470 0x7fd930157f70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9300aed50 0x7fd93014c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 --2- 172.21.15.123:0/677284526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9300ce3d0 0x7fd93015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 >> 172.21.15.123:0/677284526 conn(0x7fd9300cc380 msgr2=0x7fd9301572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 shutdown_connections 2023-06-09T04:34:53.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.305+0000 7fd935bb4700 1 -- 172.21.15.123:0/677284526 wait complete. 2023-06-09T04:34:53.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 3aac59e9-dc25-4242-8210-8ea207bec83a 2 2023-06-09T04:34:53.360 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:53.360+0000 7f0349711200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-09T04:34:53.396 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:53.394+0000 7f01cd798200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-09T04:34:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.436+0000 7f887544a700 1 Processor -- start 2023-06-09T04:34:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.436+0000 7f887544a700 1 -- start start 2023-06-09T04:34:53.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.436+0000 7f887544a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f88700cce60 0x7f887015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.436+0000 7f887544a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887015d040 con 0x7f88700cf440 2023-06-09T04:34:53.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f887544a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8870160ab0 con 0x7f8870069db0 2023-06-09T04:34:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f887544a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887015ca10 con 0x7f88700cce60 2023-06-09T04:34:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886effd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8870069db0 0x7f887006a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:50180/0 (socket says 172.21.15.123:50180) 2023-06-09T04:34:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886effd700 1 -- 172.21.15.123:0/1629849508 learned_addr learned my addr 172.21.15.123:0/1629849508 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-06-09T04:34:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886f7fe700 1 --2- 172.21.15.123:0/1629849508 >> v2:172.21.15.92:3300/0 conn(0x7f88700cce60 0x7f887015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886dffb700 1 -- 172.21.15.123:0/1629849508 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701021962 0 0) 0x7f8870160ab0 con 0x7f8870069db0 2023-06-09T04:34:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886dffb700 1 -- 172.21.15.123:0/1629849508 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88580035b0 con 0x7f8870069db0 2023-06-09T04:34:53.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886dffb700 1 -- 172.21.15.123:0/1629849508 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363009226 0 0) 0x7f887015d040 con 0x7f88700cf440 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886dffb700 1 -- 172.21.15.123:0/1629849508 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8870160ab0 con 0x7f88700cf440 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886f7fe700 1 -- 172.21.15.123:0/1629849508 >> v1:172.21.15.123:6789/0 conn(0x7f8870069db0 legacy=0x7f887006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886f7fe700 1 -- 172.21.15.123:0/1629849508 >> v1:172.21.15.92:6789/0 conn(0x7f88700cf440 legacy=0x7f887006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886f7fe700 1 -- 172.21.15.123:0/1629849508 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8870160d50 con 0x7f88700cce60 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886f7fe700 1 --2- 172.21.15.123:0/1629849508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700cce60 0x7f887015d4d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8868009d90 tx=0x7f886800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17423adf9d7c1a2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886dffb700 1 -- 172.21.15.123:0/1629849508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886800e070 con 0x7f88700cce60 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886dffb700 1 -- 172.21.15.123:0/1629849508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886800bb20 con 0x7f88700cce60 2023-06-09T04:34:53.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.437+0000 7f886dffb700 1 -- 172.21.15.123:0/1629849508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868012430 con 0x7f88700cce60 2023-06-09T04:34:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.438+0000 7f887544a700 1 -- 172.21.15.123:0/1629849508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700cce60 msgr2=0x7f887015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.438+0000 7f887544a700 1 --2- 172.21.15.123:0/1629849508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700cce60 0x7f887015d4d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8868009d90 tx=0x7f886800b330 comp rx=0 tx=0).stop 2023-06-09T04:34:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.438+0000 7f887544a700 1 -- 172.21.15.123:0/1629849508 shutdown_connections 2023-06-09T04:34:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.438+0000 7f887544a700 1 --2- 172.21.15.123:0/1629849508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88700cce60 0x7f887015d4d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.438+0000 7f887544a700 1 -- 172.21.15.123:0/1629849508 >> 172.21.15.123:0/1629849508 conn(0x7f88700cc380 msgr2=0x7f88700cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.438+0000 7f887544a700 1 -- 172.21.15.123:0/1629849508 shutdown_connections 2023-06-09T04:34:53.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.438+0000 7f887544a700 1 -- 172.21.15.123:0/1629849508 wait complete. 2023-06-09T04:34:53.443 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:53.440+0000 7f0349711200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-09T04:34:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.439+0000 7f887544a700 1 Processor -- start 2023-06-09T04:34:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f887544a700 1 -- start start 2023-06-09T04:34:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f887544a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870069db0 0x7f88700ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f887544a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88700cce60 0x7f8870151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f887544a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88700cf440 0x7f88700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f887544a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700c3ec0 con 0x7f8870069db0 2023-06-09T04:34:53.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f887544a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700c41a0 con 0x7f88700cce60 2023-06-09T04:34:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f887544a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88700c4480 con 0x7f88700cf440 2023-06-09T04:34:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f886f7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88700cf440 0x7f88700b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f886f7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88700cf440 0x7f88700b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44548/0 (socket says 172.21.15.123:44548) 2023-06-09T04:34:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f886f7fe700 1 -- 172.21.15.123:0/762228755 learned_addr learned my addr 172.21.15.123:0/762228755 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f886e7fc700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88700cce60 0x7f8870151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.440+0000 7f886effd700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870069db0 0x7f88700ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886e7fc700 1 -- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88700cf440 msgr2=0x7f88700b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886e7fc700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88700cf440 0x7f88700b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886e7fc700 1 -- 172.21.15.123:0/762228755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870069db0 msgr2=0x7f88700ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886e7fc700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870069db0 0x7f88700ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886e7fc700 1 -- 172.21.15.123:0/762228755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868009a70 con 0x7f88700cce60 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886effd700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870069db0 0x7f88700ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886f7fe700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88700cf440 0x7f88700b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f886e7fc700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88700cce60 0x7f8870151b90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f886400c3c0 tx=0x7f886400c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f88577fe700 1 -- 172.21.15.123:0/762228755 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88640052d0 con 0x7f88700cce60 2023-06-09T04:34:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f88577fe700 1 -- 172.21.15.123:0/762228755 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864005430 con 0x7f88700cce60 2023-06-09T04:34:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88700c4760 con 0x7f88700cce60 2023-06-09T04:34:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f88577fe700 1 -- 172.21.15.123:0/762228755 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88640038c0 con 0x7f88700cce60 2023-06-09T04:34:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88701520d0 con 0x7f88700cce60 2023-06-09T04:34:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f88577fe700 1 -- 172.21.15.123:0/762228755 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8864003d20 con 0x7f88700cce60 2023-06-09T04:34:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.441+0000 7f88577fe700 1 -- 172.21.15.123:0/762228755 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f8864012840 con 0x7f88700cce60 2023-06-09T04:34:53.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.442+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887006be50 con 0x7f88700cce60 2023-06-09T04:34:53.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.446+0000 7f88577fe700 1 -- 172.21.15.123:0/762228755 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8864017070 con 0x7f88700cce60 2023-06-09T04:34:53.481 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:53.480+0000 7f01cd798200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-09T04:34:53.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.528+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3aac59e9-dc25-4242-8210-8ea207bec83a", "id": 2} v 0) v1 -- 0x7f887004e8e0 con 0x7f88700cce60 2023-06-09T04:34:53.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.536+0000 7f88577fe700 1 -- 172.21.15.123:0/762228755 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3aac59e9-dc25-4242-8210-8ea207bec83a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f88640124e0 con 0x7f88700cce60 2023-06-09T04:34:53.537 INFO:teuthology.orchestra.run.smithi123.stdout:2 2023-06-09T04:34:53.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88700cce60 msgr2=0x7f8870151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88700cce60 0x7f8870151b90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f886400c3c0 tx=0x7f886400c6d0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 shutdown_connections 2023-06-09T04:34:53.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88700cf440 0x7f88700b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870069db0 0x7f88700ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 --2- 172.21.15.123:0/762228755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88700cce60 0x7f8870151b90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 >> 172.21.15.123:0/762228755 conn(0x7f88700cc380 msgr2=0x7f887006f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 shutdown_connections 2023-06-09T04:34:53.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.537+0000 7f887544a700 1 -- 172.21.15.123:0/762228755 wait complete. 2023-06-09T04:34:53.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new e4e32b61-763b-465f-92e8-77267ad85fc5 3 2023-06-09T04:34:53.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.670+0000 7f87ce5d4700 1 Processor -- start 2023-06-09T04:34:53.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.670+0000 7f87ce5d4700 1 -- start start 2023-06-09T04:34:53.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ce5d4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87c80cf440 0x7f87c815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ce5d4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ce5d4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c815d040 con 0x7f87c8069db0 2023-06-09T04:34:53.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ce5d4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c8160ab0 con 0x7f87c80cf440 2023-06-09T04:34:53.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ce5d4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c815ca10 con 0x7f87c80cce60 2023-06-09T04:34:53.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87c77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87c80cf440 0x7f87c815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87c77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87c80cf440 0x7f87c815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40860/0 (socket says 172.21.15.123:40860) 2023-06-09T04:34:53.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87c77fe700 1 -- 172.21.15.123:0/1611303031 learned_addr learned my addr 172.21.15.123:0/1611303031 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:53.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ccb71700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ccb71700 1 -- 172.21.15.123:0/1611303031 >> v2:172.21.15.123:3300/0 conn(0x7f87c80cf440 msgr2=0x7f87c815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:53.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ccb71700 1 --2- 172.21.15.123:0/1611303031 >> v2:172.21.15.123:3300/0 conn(0x7f87c80cf440 0x7f87c815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ccb71700 1 -- 172.21.15.123:0/1611303031 >> v1:172.21.15.92:6789/0 conn(0x7f87c8069db0 legacy=0x7f87c806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.680 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87ccb71700 1 -- 172.21.15.123:0/1611303031 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c8160d50 con 0x7f87c80cce60 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.671+0000 7f87c77fe700 1 --2- 172.21.15.123:0/1611303031 >> v2:172.21.15.123:3300/0 conn(0x7f87c80cf440 0x7f87c815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ccb71700 1 --2- 172.21.15.123:0/1611303031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c806ee20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009d90 tx=0x7f87c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db639a05e29f0943 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87c6ffd700 1 -- 172.21.15.123:0/1611303031 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c000e070 con 0x7f87c80cce60 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87c6ffd700 1 -- 172.21.15.123:0/1611303031 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c000bb20 con 0x7f87c80cce60 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87c6ffd700 1 -- 172.21.15.123:0/1611303031 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0012430 con 0x7f87c80cce60 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 -- 172.21.15.123:0/1611303031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 msgr2=0x7f87c806ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 --2- 172.21.15.123:0/1611303031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c806ee20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009d90 tx=0x7f87c000b330 comp rx=0 tx=0).stop 2023-06-09T04:34:53.681 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 -- 172.21.15.123:0/1611303031 shutdown_connections 2023-06-09T04:34:53.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 --2- 172.21.15.123:0/1611303031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c806ee20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 --2- 172.21.15.123:0/1611303031 >> v2:172.21.15.123:3300/0 conn(0x7f87c80cf440 0x7f87c815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 -- 172.21.15.123:0/1611303031 >> 172.21.15.123:0/1611303031 conn(0x7f87c80cc380 msgr2=0x7f87c80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 -- 172.21.15.123:0/1611303031 shutdown_connections 2023-06-09T04:34:53.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.672+0000 7f87ce5d4700 1 -- 172.21.15.123:0/1611303031 wait complete. 2023-06-09T04:34:53.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 Processor -- start 2023-06-09T04:34:53.682 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 -- start start 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87c8069db0 0x7f87c8156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c8155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87c80cf440 0x7f87c80b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80b3140 con 0x7f87c80cf440 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80b3fa0 con 0x7f87c8069db0 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.674+0000 7f87ce5d4700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c80b3390 con 0x7f87c80cce60 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c77fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c8155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87c8069db0 0x7f87c8156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87c8069db0 0x7f87c8156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40864/0 (socket says 172.21.15.123:40864) 2023-06-09T04:34:53.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c77fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c8155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44558/0 (socket says 172.21.15.123:44558) 2023-06-09T04:34:53.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 -- 172.21.15.123:0/565048984 learned_addr learned my addr 172.21.15.123:0/565048984 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:53.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87ccb71700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87c80cf440 0x7f87c80b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 -- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 msgr2=0x7f87c8155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c8155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 -- 172.21.15.123:0/565048984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87c80cf440 msgr2=0x7f87c80b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87c80cf440 0x7f87c80b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 -- 172.21.15.123:0/565048984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc009550 con 0x7f87c8069db0 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c77fe700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c8155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c7fff700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87c8069db0 0x7f87c8156040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c4e0 tx=0x7f87b800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c4ff9700 1 -- 172.21.15.123:0/565048984 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80052d0 con 0x7f87c8069db0 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c4ff9700 1 -- 172.21.15.123:0/565048984 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8005430 con 0x7f87c8069db0 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c0009a70 con 0x7f87c8069db0 2023-06-09T04:34:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c4ff9700 1 -- 172.21.15.123:0/565048984 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b80038c0 con 0x7f87c8069db0 2023-06-09T04:34:53.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c80b3950 con 0x7f87c8069db0 2023-06-09T04:34:53.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c4ff9700 1 -- 172.21.15.123:0/565048984 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f87b8003d20 con 0x7f87c8069db0 2023-06-09T04:34:53.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.675+0000 7f87c4ff9700 1 -- 172.21.15.123:0/565048984 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f87b8011d60 con 0x7f87c8069db0 2023-06-09T04:34:53.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.676+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c806be50 con 0x7f87c8069db0 2023-06-09T04:34:53.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.680+0000 7f87c4ff9700 1 -- 172.21.15.123:0/565048984 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f87b8017070 con 0x7f87c8069db0 2023-06-09T04:34:53.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.762+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e4e32b61-763b-465f-92e8-77267ad85fc5", "id": 3} v 0) v1 -- 0x7f87c804e8e0 con 0x7f87c8069db0 2023-06-09T04:34:53.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.770+0000 7f87c4ff9700 1 -- 172.21.15.123:0/565048984 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e4e32b61-763b-465f-92e8-77267ad85fc5", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f87b8011590 con 0x7f87c8069db0 2023-06-09T04:34:53.771 INFO:teuthology.orchestra.run.smithi123.stdout:3 2023-06-09T04:34:53.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87c8069db0 msgr2=0x7f87c8156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87c8069db0 0x7f87c8156040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c4e0 tx=0x7f87b800c7f0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 shutdown_connections 2023-06-09T04:34:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87c80cf440 0x7f87c80b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87c8069db0 0x7f87c8156040 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 --2- 172.21.15.123:0/565048984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87c80cce60 0x7f87c8155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 >> 172.21.15.123:0/565048984 conn(0x7f87c80cc380 msgr2=0x7f87c80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 shutdown_connections 2023-06-09T04:34:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.771+0000 7f87ce5d4700 1 -- 172.21.15.123:0/565048984 wait complete. 2023-06-09T04:34:53.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 69d5fe7c-90b1-434f-afaa-90d341672a14 4 2023-06-09T04:34:53.872 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:53.871+0000 7f0349711200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-09T04:34:53.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.900+0000 7f8bcbafa700 1 Processor -- start 2023-06-09T04:34:53.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.900+0000 7f8bcbafa700 1 -- start start 2023-06-09T04:34:53.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bcbafa700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8bc40aea90 0x7f8bc40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bcbafa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc406a4c0 0x7f8bc406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bcbafa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc4071dc0 con 0x7f8bc40aee60 2023-06-09T04:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bcbafa700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc415edf0 con 0x7f8bc40aea90 2023-06-09T04:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bcbafa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40cf030 con 0x7f8bc406a4c0 2023-06-09T04:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bc9896700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8bc40aea90 0x7f8bc40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bca097700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc406a4c0 0x7f8bc406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bca097700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc406a4c0 0x7f8bc406ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44566/0 (socket says 172.21.15.123:44566) 2023-06-09T04:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bc9896700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8bc40aea90 0x7f8bc40cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40870/0 (socket says 172.21.15.123:40870) 2023-06-09T04:34:53.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.901+0000 7f8bca097700 1 -- 172.21.15.123:0/4259020869 learned_addr learned my addr 172.21.15.123:0/4259020869 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bca097700 1 -- 172.21.15.123:0/4259020869 >> v2:172.21.15.123:3300/0 conn(0x7f8bc40aea90 msgr2=0x7f8bc40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bca097700 1 --2- 172.21.15.123:0/4259020869 >> v2:172.21.15.123:3300/0 conn(0x7f8bc40aea90 0x7f8bc40cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bca097700 1 -- 172.21.15.123:0/4259020869 >> v1:172.21.15.92:6789/0 conn(0x7f8bc40aee60 legacy=0x7f8bc4069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bca097700 1 -- 172.21.15.123:0/4259020869 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40cf2d0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bc9896700 1 --2- 172.21.15.123:0/4259020869 >> v2:172.21.15.123:3300/0 conn(0x7f8bc40aea90 0x7f8bc40cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bca097700 1 --2- 172.21.15.123:0/4259020869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc406a4c0 0x7f8bc406ee20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009e50 tx=0x7f8bbc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129dc5229d7bf838 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bc8894700 1 -- 172.21.15.123:0/4259020869 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00e070 con 0x7f8bc406a4c0 2023-06-09T04:34:53.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bc8894700 1 -- 172.21.15.123:0/4259020869 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc00b910 con 0x7f8bc406a4c0 2023-06-09T04:34:53.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.902+0000 7f8bc8894700 1 -- 172.21.15.123:0/4259020869 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00bcc0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 -- 172.21.15.123:0/4259020869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc406a4c0 msgr2=0x7f8bc406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 --2- 172.21.15.123:0/4259020869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc406a4c0 0x7f8bc406ee20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009e50 tx=0x7f8bbc00b330 comp rx=0 tx=0).stop 2023-06-09T04:34:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 -- 172.21.15.123:0/4259020869 shutdown_connections 2023-06-09T04:34:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 --2- 172.21.15.123:0/4259020869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc406a4c0 0x7f8bc406ee20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 --2- 172.21.15.123:0/4259020869 >> v2:172.21.15.123:3300/0 conn(0x7f8bc40aea90 0x7f8bc40cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 -- 172.21.15.123:0/4259020869 >> 172.21.15.123:0/4259020869 conn(0x7f8bc40cc380 msgr2=0x7f8bc40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 -- 172.21.15.123:0/4259020869 shutdown_connections 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.903+0000 7f8bcbafa700 1 -- 172.21.15.123:0/4259020869 wait complete. 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.904+0000 7f8bcbafa700 1 Processor -- start 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.904+0000 7f8bcbafa700 1 -- start start 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc406a4c0 0x7f8bc414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc40aea90 0x7f8bc40ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc40cb840 0x7f8bc40c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4150650 con 0x7f8bc40aea90 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40cb590 con 0x7f8bc406a4c0 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40cbcc0 con 0x7f8bc40cb840 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc406a4c0 0x7f8bc414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc406a4c0 0x7f8bc414cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40876/0 (socket says 172.21.15.123:40876) 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 -- 172.21.15.123:0/1335811228 learned_addr learned my addr 172.21.15.123:0/1335811228 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bca097700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc40cb840 0x7f8bc40c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9095700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc40aea90 0x7f8bc40ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 -- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc40cb840 msgr2=0x7f8bc40c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc40cb840 0x7f8bc40c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 -- 172.21.15.123:0/1335811228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc40aea90 msgr2=0x7f8bc40ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc40aea90 0x7f8bc40ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 -- 172.21.15.123:0/1335811228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00094a0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bc9896700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc406a4c0 0x7f8bc414cf00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000ec10 tx=0x7f8bc000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:53.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bba7fc700 1 -- 172.21.15.123:0/1335811228 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000c7f0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bba7fc700 1 -- 172.21.15.123:0/1335811228 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc000c950 con 0x7f8bc406a4c0 2023-06-09T04:34:53.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc009b30 con 0x7f8bc406a4c0 2023-06-09T04:34:53.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bba7fc700 1 -- 172.21.15.123:0/1335811228 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000cce0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.905+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc40abde0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.906+0000 7f8bba7fc700 1 -- 172.21.15.123:0/1335811228 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8bc0013770 con 0x7f8bc406a4c0 2023-06-09T04:34:53.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.906+0000 7f8bba7fc700 1 -- 172.21.15.123:0/1335811228 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f8bc001cc80 con 0x7f8bc406a4c0 2023-06-09T04:34:53.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.906+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc404e8e0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.908+0000 7f8bba7fc700 1 -- 172.21.15.123:0/1335811228 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8bc003bb00 con 0x7f8bc406a4c0 2023-06-09T04:34:53.956 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:53.955+0000 7f01cd798200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-09T04:34:53.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.990+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "69d5fe7c-90b1-434f-afaa-90d341672a14", "id": 4} v 0) v1 -- 0x7f8bc406a9b0 con 0x7f8bc406a4c0 2023-06-09T04:34:53.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.998+0000 7f8bba7fc700 1 -- 172.21.15.123:0/1335811228 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69d5fe7c-90b1-434f-afaa-90d341672a14", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8bc002a030 con 0x7f8bc406a4c0 2023-06-09T04:34:54.000 INFO:teuthology.orchestra.run.smithi123.stdout:4 2023-06-09T04:34:54.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc406a4c0 msgr2=0x7f8bc414cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc406a4c0 0x7f8bc414cf00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000ec10 tx=0x7f8bc000ef20 comp rx=0 tx=0).stop 2023-06-09T04:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 shutdown_connections 2023-06-09T04:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bc40cb840 0x7f8bc40c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bc406a4c0 0x7f8bc414cf00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 --2- 172.21.15.123:0/1335811228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bc40aea90 0x7f8bc40ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 >> 172.21.15.123:0/1335811228 conn(0x7f8bc40cc380 msgr2=0x7f8bc40b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 shutdown_connections 2023-06-09T04:34:54.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:53.999+0000 7f8bcbafa700 1 -- 172.21.15.123:0/1335811228 wait complete. 2023-06-09T04:34:54.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new ad7f134c-14ea-49aa-9bcb-b0c46898834d 5 2023-06-09T04:34:54.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.132+0000 7fdcf4515700 1 Processor -- start 2023-06-09T04:34:54.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.132+0000 7fdcf4515700 1 -- start start 2023-06-09T04:34:54.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf4515700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcec0aea90 0x7fdcec0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf4515700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdcec0aee60 0x7fdcec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf4515700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec071dc0 con 0x7fdcec15e840 2023-06-09T04:34:54.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf4515700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcec15edf0 con 0x7fdcec0aee60 2023-06-09T04:34:54.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf4515700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcec06fe30 con 0x7fdcec0aea90 2023-06-09T04:34:54.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf22b1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcec0aea90 0x7fdcec0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf22b1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcec0aea90 0x7fdcec0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40882/0 (socket says 172.21.15.123:40882) 2023-06-09T04:34:54.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf22b1700 1 -- 172.21.15.123:0/3138667414 learned_addr learned my addr 172.21.15.123:0/3138667414 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:54.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.133+0000 7fdcf1ab0700 1 --2- 172.21.15.123:0/3138667414 >> v2:172.21.15.92:3300/0 conn(0x7fdcec0aee60 0x7fdcec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf22b1700 1 -- 172.21.15.123:0/3138667414 >> v2:172.21.15.92:3300/0 conn(0x7fdcec0aee60 msgr2=0x7fdcec069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:54.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf22b1700 1 --2- 172.21.15.123:0/3138667414 >> v2:172.21.15.92:3300/0 conn(0x7fdcec0aee60 0x7fdcec069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf22b1700 1 -- 172.21.15.123:0/3138667414 >> v1:172.21.15.123:6789/0 conn(0x7fdcec15e840 legacy=0x7fdcec0b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf22b1700 1 -- 172.21.15.123:0/3138667414 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0cf030 con 0x7fdcec0aea90 2023-06-09T04:34:54.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf1ab0700 1 --2- 172.21.15.123:0/3138667414 >> v2:172.21.15.92:3300/0 conn(0x7fdcec0aee60 0x7fdcec069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:34:54.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf22b1700 1 --2- 172.21.15.123:0/3138667414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0aea90 0x7fdcec0cce60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0710a0 tx=0x7fdce8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e55bf6125ed297 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf12af700 1 -- 172.21.15.123:0/3138667414 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800b870 con 0x7fdcec0aea90 2023-06-09T04:34:54.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf12af700 1 -- 172.21.15.123:0/3138667414 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce800b9d0 con 0x7fdcec0aea90 2023-06-09T04:34:54.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf12af700 1 -- 172.21.15.123:0/3138667414 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce800bd60 con 0x7fdcec0aea90 2023-06-09T04:34:54.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf4515700 1 -- 172.21.15.123:0/3138667414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0aea90 msgr2=0x7fdcec0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf4515700 1 --2- 172.21.15.123:0/3138667414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0aea90 0x7fdcec0cce60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0710a0 tx=0x7fdce8009bd0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf4515700 1 -- 172.21.15.123:0/3138667414 shutdown_connections 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf4515700 1 --2- 172.21.15.123:0/3138667414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0aea90 0x7fdcec0cce60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf4515700 1 --2- 172.21.15.123:0/3138667414 >> v2:172.21.15.92:3300/0 conn(0x7fdcec0aee60 0x7fdcec069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.134+0000 7fdcf4515700 1 -- 172.21.15.123:0/3138667414 >> 172.21.15.123:0/3138667414 conn(0x7fdcec0cc380 msgr2=0x7fdcec0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.135+0000 7fdcf4515700 1 -- 172.21.15.123:0/3138667414 shutdown_connections 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.135+0000 7fdcf4515700 1 -- 172.21.15.123:0/3138667414 wait complete. 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 Processor -- start 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 -- start start 2023-06-09T04:34:54.141 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcec0aea90 0x7fdcec158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec15e840 0x7fdcec0adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0c9f50 0x7fdcec0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0ae180 con 0x7fdcec0aea90 2023-06-09T04:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0ca6f0 con 0x7fdcec0c9f50 2023-06-09T04:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.136+0000 7fdcf4515700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0ca9d0 con 0x7fdcec15e840 2023-06-09T04:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec15e840 0x7fdcec0adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec15e840 0x7fdcec0adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44582/0 (socket says 172.21.15.123:44582) 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 -- 172.21.15.123:0/707133062 learned_addr learned my addr 172.21.15.123:0/707133062 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf22b1700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcec0aea90 0x7fdcec158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf2ab2700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0c9f50 0x7fdcec0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 -- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0c9f50 msgr2=0x7fdcec0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0c9f50 0x7fdcec0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 -- 172.21.15.123:0/707133062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcec0aea90 msgr2=0x7fdcec158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcec0aea90 0x7fdcec158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 -- 172.21.15.123:0/707133062 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce8009550 con 0x7fdcec15e840 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf2ab2700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0c9f50 0x7fdcec0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf22b1700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcec0aea90 0x7fdcec158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf1ab0700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec15e840 0x7fdcec0adc40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00e7e0 tx=0x7fdcdc00eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdce2ffd700 1 -- 172.21.15.123:0/707133062 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00c070 con 0x7fdcec15e840 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdce2ffd700 1 -- 172.21.15.123:0/707133062 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc004230 con 0x7fdcec15e840 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdce2ffd700 1 -- 172.21.15.123:0/707133062 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0045a0 con 0x7fdcec15e840 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec0cacb0 con 0x7fdcec15e840 2023-06-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec0c7af0 con 0x7fdcec15e840 2023-06-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdce2ffd700 1 -- 172.21.15.123:0/707133062 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdcdc0039a0 con 0x7fdcec15e840 2023-06-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.137+0000 7fdce2ffd700 1 -- 172.21.15.123:0/707133062 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fdcdc013b90 con 0x7fdcec15e840 2023-06-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.138+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec0c7950 con 0x7fdcec15e840 2023-06-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.141+0000 7fdce2ffd700 1 -- 172.21.15.123:0/707133062 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fdcec0c7950 con 0x7fdcec15e840 2023-06-09T04:34:54.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.222+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ad7f134c-14ea-49aa-9bcb-b0c46898834d", "id": 5} v 0) v1 -- 0x7fdcec04e8e0 con 0x7fdcec15e840 2023-06-09T04:34:54.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.230+0000 7fdce2ffd700 1 -- 172.21.15.123:0/707133062 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ad7f134c-14ea-49aa-9bcb-b0c46898834d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdcdc00f040 con 0x7fdcec15e840 2023-06-09T04:34:54.232 INFO:teuthology.orchestra.run.smithi123.stdout:5 2023-06-09T04:34:54.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.231+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec15e840 msgr2=0x7fdcec0adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.231+0000 7fdcf4515700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec15e840 0x7fdcec0adc40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00e7e0 tx=0x7fdcdc00eaf0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.231+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 shutdown_connections 2023-06-09T04:34:54.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.231+0000 7fdcf4515700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcec0c9f50 0x7fdcec0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.231+0000 7fdcf4515700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcec0aea90 0x7fdcec158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.231+0000 7fdcf4515700 1 --2- 172.21.15.123:0/707133062 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcec15e840 0x7fdcec0adc40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.231+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 >> 172.21.15.123:0/707133062 conn(0x7fdcec0cc380 msgr2=0x7fdcec0ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.232+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 shutdown_connections 2023-06-09T04:34:54.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.232+0000 7fdcf4515700 1 -- 172.21.15.123:0/707133062 wait complete. 2023-06-09T04:34:54.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new ec5c5ee3-4cc5-4751-8576-2e584f15095d 6 2023-06-09T04:34:54.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.377+0000 7fedaa81b700 1 Processor -- start 2023-06-09T04:34:54.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.377+0000 7fedaa81b700 1 -- start start 2023-06-09T04:34:54.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.378+0000 7fedaa81b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7feda40cb8d0 0x7feda406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.378+0000 7fedaa81b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feda406de30 0x7feda40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.378+0000 7fedaa81b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda40c8f30 con 0x7feda4158670 2023-06-09T04:34:54.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.378+0000 7fedaa81b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda4158a40 con 0x7feda40cb8d0 2023-06-09T04:34:54.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.378+0000 7fedaa81b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda406c9c0 con 0x7feda406de30 2023-06-09T04:34:54.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feda406de30 0x7feda40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feda406de30 0x7feda40c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40896/0 (socket says 172.21.15.123:40896) 2023-06-09T04:34:54.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 -- 172.21.15.123:0/3376377103 learned_addr learned my addr 172.21.15.123:0/3376377103 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:54.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda37fe700 1 --2- 172.21.15.123:0/3376377103 >> v2:172.21.15.92:3300/0 conn(0x7feda40cb8d0 0x7feda406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 -- 172.21.15.123:0/3376377103 >> v2:172.21.15.92:3300/0 conn(0x7feda40cb8d0 msgr2=0x7feda406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:54.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 --2- 172.21.15.123:0/3376377103 >> v2:172.21.15.92:3300/0 conn(0x7feda40cb8d0 0x7feda406c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 -- 172.21.15.123:0/3376377103 >> v1:172.21.15.123:6789/0 conn(0x7feda4158670 legacy=0x7feda40cb1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:34:54.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 -- 172.21.15.123:0/3376377103 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda406cca0 con 0x7feda406de30 2023-06-09T04:34:54.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.379+0000 7feda8db8700 1 --2- 172.21.15.123:0/3376377103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c93c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feda406b6b0 tx=0x7fed9c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb188f662691319 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7feda2ffd700 1 -- 172.21.15.123:0/3376377103 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c004230 con 0x7feda406de30 2023-06-09T04:34:54.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7feda2ffd700 1 -- 172.21.15.123:0/3376377103 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c004390 con 0x7feda406de30 2023-06-09T04:34:54.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 -- 172.21.15.123:0/3376377103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 msgr2=0x7feda40c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 --2- 172.21.15.123:0/3376377103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c93c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feda406b6b0 tx=0x7fed9c009bd0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 -- 172.21.15.123:0/3376377103 shutdown_connections 2023-06-09T04:34:54.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 --2- 172.21.15.123:0/3376377103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c93c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 --2- 172.21.15.123:0/3376377103 >> v2:172.21.15.92:3300/0 conn(0x7feda40cb8d0 0x7feda406c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 -- 172.21.15.123:0/3376377103 >> 172.21.15.123:0/3376377103 conn(0x7feda406aa40 msgr2=0x7feda40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 -- 172.21.15.123:0/3376377103 shutdown_connections 2023-06-09T04:34:54.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.380+0000 7fedaa81b700 1 -- 172.21.15.123:0/3376377103 wait complete. 2023-06-09T04:34:54.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.382+0000 7fedaa81b700 1 Processor -- start 2023-06-09T04:34:54.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.382+0000 7fedaa81b700 1 -- start start 2023-06-09T04:34:54.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.383+0000 7fedaa81b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.383+0000 7fedaa81b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feda40cb8d0 0x7feda40c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.383+0000 7fedaa81b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda4158670 0x7feda40c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.383+0000 7fedaa81b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda4148da0 con 0x7feda4158670 2023-06-09T04:34:54.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.383+0000 7fedaa81b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda4149d70 con 0x7feda406de30 2023-06-09T04:34:54.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.383+0000 7fedaa81b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda41498e0 con 0x7feda40cb8d0 2023-06-09T04:34:54.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feda40cb8d0 0x7feda40c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feda40cb8d0 0x7feda40c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44590/0 (socket says 172.21.15.123:44590) 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda37fe700 1 -- 172.21.15.123:0/543393345 learned_addr learned my addr 172.21.15.123:0/543393345 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda3fff700 1 -- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feda40cb8d0 msgr2=0x7feda40c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda3fff700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feda40cb8d0 0x7feda40c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda3fff700 1 -- 172.21.15.123:0/543393345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda4158670 msgr2=0x7feda40c7f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda3fff700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda4158670 0x7feda40c7f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda3fff700 1 -- 172.21.15.123:0/543393345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c009550 con 0x7feda406de30 2023-06-09T04:34:54.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda37fe700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feda40cb8d0 0x7feda40c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:54.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.384+0000 7feda3fff700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c4c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00b3e0 tx=0x7fed8c00b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.385+0000 7feda0ff9700 1 -- 172.21.15.123:0/543393345 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c011830 con 0x7feda406de30 2023-06-09T04:34:54.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.385+0000 7feda0ff9700 1 -- 172.21.15.123:0/543393345 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c011990 con 0x7feda406de30 2023-06-09T04:34:54.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.385+0000 7feda0ff9700 1 -- 172.21.15.123:0/543393345 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c011d40 con 0x7feda406de30 2023-06-09T04:34:54.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.385+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda4148fb0 con 0x7feda406de30 2023-06-09T04:34:54.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.385+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda4149570 con 0x7feda406de30 2023-06-09T04:34:54.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.385+0000 7feda0ff9700 1 -- 172.21.15.123:0/543393345 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fed8c01a700 con 0x7feda406de30 2023-06-09T04:34:54.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.385+0000 7feda0ff9700 1 -- 172.21.15.123:0/543393345 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fed8c0194f0 con 0x7feda406de30 2023-06-09T04:34:54.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.386+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda4149730 con 0x7feda406de30 2023-06-09T04:34:54.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.389+0000 7feda0ff9700 1 -- 172.21.15.123:0/543393345 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fed8c039b00 con 0x7feda406de30 2023-06-09T04:34:54.436 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:54.435+0000 7f0349711200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-09T04:34:54.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.483+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ec5c5ee3-4cc5-4751-8576-2e584f15095d", "id": 6} v 0) v1 -- 0x7feda404e8e0 con 0x7feda406de30 2023-06-09T04:34:54.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.494+0000 7feda0ff9700 1 -- 172.21.15.123:0/543393345 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec5c5ee3-4cc5-4751-8576-2e584f15095d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fed8c028030 con 0x7feda406de30 2023-06-09T04:34:54.495 INFO:teuthology.orchestra.run.smithi123.stdout:6 2023-06-09T04:34:54.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 msgr2=0x7feda40c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c4c60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00b3e0 tx=0x7fed8c00b6f0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 shutdown_connections 2023-06-09T04:34:54.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feda4158670 0x7feda40c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feda406de30 0x7feda40c4c60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 --2- 172.21.15.123:0/543393345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feda40cb8d0 0x7feda40c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 >> 172.21.15.123:0/543393345 conn(0x7feda406aa40 msgr2=0x7feda40cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 shutdown_connections 2023-06-09T04:34:54.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.495+0000 7fedaa81b700 1 -- 172.21.15.123:0/543393345 wait complete. 2023-06-09T04:34:54.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 2606c92d-19ca-4b65-b058-ea5e8de3c0a3 7 2023-06-09T04:34:54.537 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:54.536+0000 7f01cd798200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-09T04:34:54.591 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:54.590+0000 7f0349711200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-09T04:34:54.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.627+0000 7f89f81ab700 1 Processor -- start 2023-06-09T04:34:54.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.628+0000 7f89f81ab700 1 -- start start 2023-06-09T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.628+0000 7f89f81ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.628+0000 7f89f81ab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89f00b2930 0x7f89f00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.628+0000 7f89f81ab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f015d270 con 0x7f89f01569f0 2023-06-09T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.628+0000 7f89f81ab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00b2db0 con 0x7f89f00b2930 2023-06-09T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.628+0000 7f89f81ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f015d550 con 0x7f89f00b2f90 2023-06-09T04:34:54.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89f00b2930 0x7f89f00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f5746700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89f00b2930 0x7f89f00ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40908/0 (socket says 172.21.15.123:40908) 2023-06-09T04:34:54.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f5746700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44598/0 (socket says 172.21.15.123:44598) 2023-06-09T04:34:54.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 -- 172.21.15.123:0/3743794300 learned_addr learned my addr 172.21.15.123:0/3743794300 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:54.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 -- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 msgr2=0x7f89f00b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:54.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 --2- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 -- 172.21.15.123:0/3743794300 >> v1:172.21.15.123:6789/0 conn(0x7f89f01569f0 legacy=0x7f89f015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 -- 172.21.15.123:0/3743794300 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f015d830 con 0x7f89f00b2930 2023-06-09T04:34:54.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f4f45700 1 -- 172.21.15.123:0/3743794300 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353956343 0 0) 0x7f89f015d270 con 0x7f89f01569f0 2023-06-09T04:34:54.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f5746700 1 --2- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:54.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f6748700 1 --2- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00b2930 0x7f89f00ccfe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009e50 tx=0x7f89e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e687b0e592c25171 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f4f45700 1 -- 172.21.15.123:0/3743794300 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800e070 con 0x7f89f00b2930 2023-06-09T04:34:54.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f4f45700 1 -- 172.21.15.123:0/3743794300 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e800b910 con 0x7f89f00b2930 2023-06-09T04:34:54.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.629+0000 7f89f4f45700 1 -- 172.21.15.123:0/3743794300 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800bcc0 con 0x7f89f00b2930 2023-06-09T04:34:54.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 -- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00b2930 msgr2=0x7f89f00ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 --2- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00b2930 0x7f89f00ccfe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009e50 tx=0x7f89e800b330 comp rx=0 tx=0).stop 2023-06-09T04:34:54.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 -- 172.21.15.123:0/3743794300 shutdown_connections 2023-06-09T04:34:54.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 --2- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f00b2930 0x7f89f00ccfe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 --2- 172.21.15.123:0/3743794300 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 -- 172.21.15.123:0/3743794300 >> 172.21.15.123:0/3743794300 conn(0x7f89f006e4d0 msgr2=0x7f89f006e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 -- 172.21.15.123:0/3743794300 shutdown_connections 2023-06-09T04:34:54.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.630+0000 7f89f81ab700 1 -- 172.21.15.123:0/3743794300 wait complete. 2023-06-09T04:34:54.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.631+0000 7f89f81ab700 1 Processor -- start 2023-06-09T04:34:54.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.631+0000 7f89f81ab700 1 -- start start 2023-06-09T04:34:54.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f81ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f00b2930 0x7f89f0158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f81ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f81ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f01569f0 0x7f89f01f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f81ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00b6540 con 0x7f89f00b2930 2023-06-09T04:34:54.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f81ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0153670 con 0x7f89f01569f0 2023-06-09T04:34:54.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f81ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f01f2d60 con 0x7f89f00b2f90 2023-06-09T04:34:54.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f01569f0 0x7f89f01f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f5746700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f01569f0 0x7f89f01f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:40914/0 (socket says 172.21.15.123:40914) 2023-06-09T04:34:54.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 -- 172.21.15.123:0/2981325745 learned_addr learned my addr 172.21.15.123:0/2981325745 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-06-09T04:34:54.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f5746700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:44604/0 (socket says 172.21.15.123:44604) 2023-06-09T04:34:54.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f5f47700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f00b2930 0x7f89f0158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 -- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 msgr2=0x7f89f00b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 -- 172.21.15.123:0/2981325745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f00b2930 msgr2=0x7f89f0158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f00b2930 0x7f89f0158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f6748700 1 -- 172.21.15.123:0/2981325745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e00094a0 con 0x7f89f01569f0 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f5f47700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f00b2930 0x7f89f0158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.632+0000 7f89f5746700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89f6748700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f01569f0 0x7f89f01f2820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f89e800b680 tx=0x7f89e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89e67fc700 1 -- 172.21.15.123:0/2981325745 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8004960 con 0x7f89f01569f0 2023-06-09T04:34:54.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89e67fc700 1 -- 172.21.15.123:0/2981325745 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e80099b0 con 0x7f89f01569f0 2023-06-09T04:34:54.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e8009b30 con 0x7f89f01569f0 2023-06-09T04:34:54.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89e67fc700 1 -- 172.21.15.123:0/2981325745 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e801c660 con 0x7f89f01569f0 2023-06-09T04:34:54.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f0158c60 con 0x7f89f01569f0 2023-06-09T04:34:54.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89e67fc700 1 -- 172.21.15.123:0/2981325745 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f89e801cac0 con 0x7f89f01569f0 2023-06-09T04:34:54.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.633+0000 7f89e67fc700 1 -- 172.21.15.123:0/2981325745 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f89e8012a50 con 0x7f89f01569f0 2023-06-09T04:34:54.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.634+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f004e8e0 con 0x7f89f01569f0 2023-06-09T04:34:54.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.636+0000 7f89e67fc700 1 -- 172.21.15.123:0/2981325745 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f89e8020070 con 0x7f89f01569f0 2023-06-09T04:34:54.697 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:54.696+0000 7f01cd798200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-09T04:34:54.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.719+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2606c92d-19ca-4b65-b058-ea5e8de3c0a3", "id": 7} v 0) v1 -- 0x7f89f0070890 con 0x7f89f01569f0 2023-06-09T04:34:54.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.727+0000 7f89e67fc700 1 -- 172.21.15.123:0/2981325745 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2606c92d-19ca-4b65-b058-ea5e8de3c0a3", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f89e8012cf0 con 0x7f89f01569f0 2023-06-09T04:34:54.729 INFO:teuthology.orchestra.run.smithi123.stdout:7 2023-06-09T04:34:54.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f01569f0 msgr2=0x7f89f01f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f01569f0 0x7f89f01f2820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f89e800b680 tx=0x7f89e8004ad0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 shutdown_connections 2023-06-09T04:34:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f01569f0 0x7f89f01f2820 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f00b2930 0x7f89f0158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 --2- 172.21.15.123:0/2981325745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89f00b2f90 0x7f89f00b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 >> 172.21.15.123:0/2981325745 conn(0x7f89f006e4d0 msgr2=0x7f89f00b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 shutdown_connections 2023-06-09T04:34:54.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-06-09T04:34:54.728+0000 7f89f81ab700 1 -- 172.21.15.123:0/2981325745 wait complete. 2023-06-09T04:34:54.741 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-09T04:34:54.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-09T04:34:54.743 INFO:tasks.ceph.osd.0:Started 2023-06-09T04:34:54.743 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-09T04:34:54.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-09T04:34:54.745 INFO:tasks.ceph.osd.1:Started 2023-06-09T04:34:54.746 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-09T04:34:54.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-09T04:34:54.747 INFO:tasks.ceph.osd.2:Started 2023-06-09T04:34:54.748 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-09T04:34:54.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-09T04:34:54.750 INFO:tasks.ceph.osd.3:Started 2023-06-09T04:34:54.750 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-09T04:34:54.750 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-09T04:34:54.783 INFO:tasks.ceph.osd.4:Started 2023-06-09T04:34:54.783 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-09T04:34:54.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-09T04:34:54.785 INFO:tasks.ceph.osd.5:Started 2023-06-09T04:34:54.785 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-09T04:34:54.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-09T04:34:54.787 INFO:tasks.ceph.osd.6:Started 2023-06-09T04:34:54.788 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-09T04:34:54.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-09T04:34:54.790 INFO:tasks.ceph.osd.7:Started 2023-06-09T04:34:54.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:34:54.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.946+0000 7fe88362a700 1 Processor -- start 2023-06-09T04:34:54.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.946+0000 7fe88362a700 1 -- start start 2023-06-09T04:34:54.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe88362a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe87c0ae630 0x7fe87c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe88362a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87c0adde0 con 0x7fe87c0b1360 2023-06-09T04:34:54.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe88362a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87c0af0f0 con 0x7fe87c0cd7e0 2023-06-09T04:34:54.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe88362a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87c0ae270 con 0x7fe87c0ae630 2023-06-09T04:34:54.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe8813c6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe87c0b1360 0x7fe87c0b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35188/0 (socket says 172.21.15.92:35188) 2023-06-09T04:34:54.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe8813c6700 1 -- 172.21.15.92:0/2757172740 learned_addr learned my addr 172.21.15.92:0/2757172740 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:34:54.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe880bc5700 1 --2- 172.21.15.92:0/2757172740 >> v2:172.21.15.123:3300/0 conn(0x7fe87c0ae630 0x7fe87c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe873fff700 1 -- 172.21.15.92:0/2757172740 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920984579 0 0) 0x7fe87c0adde0 con 0x7fe87c0b1360 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe873fff700 1 -- 172.21.15.92:0/2757172740 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8680035b0 con 0x7fe87c0b1360 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe880bc5700 1 -- 172.21.15.92:0/2757172740 >> v1:172.21.15.123:6789/0 conn(0x7fe87c0cd7e0 legacy=0x7fe87c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe880bc5700 1 -- 172.21.15.92:0/2757172740 >> v1:172.21.15.92:6789/0 conn(0x7fe87c0b1360 legacy=0x7fe87c0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe880bc5700 1 -- 172.21.15.92:0/2757172740 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c0ad4b0 con 0x7fe87c0ae630 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe880bc5700 1 --2- 172.21.15.92:0/2757172740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0ae630 0x7fe87c0aeaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009840 tx=0x7fe86c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede90dea8780cb75 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe873fff700 1 -- 172.21.15.92:0/2757172740 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c011830 con 0x7fe87c0ae630 2023-06-09T04:34:54.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.947+0000 7fe873fff700 1 -- 172.21.15.92:0/2757172740 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c011990 con 0x7fe87c0ae630 2023-06-09T04:34:54.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe873fff700 1 -- 172.21.15.92:0/2757172740 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c011d20 con 0x7fe87c0ae630 2023-06-09T04:34:54.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe88362a700 1 -- 172.21.15.92:0/2757172740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0ae630 msgr2=0x7fe87c0aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe88362a700 1 --2- 172.21.15.92:0/2757172740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0ae630 0x7fe87c0aeaa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009840 tx=0x7fe86c009b50 comp rx=0 tx=0).stop 2023-06-09T04:34:54.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe88362a700 1 -- 172.21.15.92:0/2757172740 shutdown_connections 2023-06-09T04:34:54.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe88362a700 1 --2- 172.21.15.92:0/2757172740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0ae630 0x7fe87c0aeaa0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe88362a700 1 -- 172.21.15.92:0/2757172740 >> 172.21.15.92:0/2757172740 conn(0x7fe87c098c10 msgr2=0x7fe87c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:54.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe88362a700 1 -- 172.21.15.92:0/2757172740 shutdown_connections 2023-06-09T04:34:54.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.948+0000 7fe88362a700 1 -- 172.21.15.92:0/2757172740 wait complete. 2023-06-09T04:34:54.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.950+0000 7fe88362a700 1 Processor -- start 2023-06-09T04:34:54.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.950+0000 7fe88362a700 1 -- start start 2023-06-09T04:34:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe88362a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe87c0ae630 0x7fe87c0c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe88362a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0b1360 0x7fe87c15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe88362a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe87c0cd7e0 0x7fe87c06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:54.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe88362a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c15a800 con 0x7fe87c0ae630 2023-06-09T04:34:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe88362a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c15c960 con 0x7fe87c0b1360 2023-06-09T04:34:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe88362a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0c3b50 con 0x7fe87c0cd7e0 2023-06-09T04:34:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe87c0ae630 0x7fe87c0c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe87c0ae630 0x7fe87c0c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36862/0 (socket says 172.21.15.92:36862) 2023-06-09T04:34:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 -- 172.21.15.92:0/783260888 learned_addr learned my addr 172.21.15.92:0/783260888 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:34:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe880bc5700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0b1360 0x7fe87c15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe881bc7700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe87c0cd7e0 0x7fe87c06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 -- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe87c0cd7e0 msgr2=0x7fe87c06cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe87c0cd7e0 0x7fe87c06cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 -- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0b1360 msgr2=0x7fe87c15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0b1360 0x7fe87c15bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 -- 172.21.15.92:0/783260888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c0094a0 con 0x7fe87c0ae630 2023-06-09T04:34:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe8813c6700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe87c0ae630 0x7fe87c0c5d80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe87800c700 tx=0x7fe87800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:54.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe871ffb700 1 -- 172.21.15.92:0/783260888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8780074e0 con 0x7fe87c0ae630 2023-06-09T04:34:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe871ffb700 1 -- 172.21.15.92:0/783260888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe878007640 con 0x7fe87c0ae630 2023-06-09T04:34:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c15c680 con 0x7fe87c0ae630 2023-06-09T04:34:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.951+0000 7fe871ffb700 1 -- 172.21.15.92:0/783260888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe878005a50 con 0x7fe87c0ae630 2023-06-09T04:34:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.952+0000 7fe871ffb700 1 -- 172.21.15.92:0/783260888 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe878012400 con 0x7fe87c0ae630 2023-06-09T04:34:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.952+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c0c37e0 con 0x7fe87c0ae630 2023-06-09T04:34:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.952+0000 7fe871ffb700 1 -- 172.21.15.92:0/783260888 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe87801b3b0 con 0x7fe87c0ae630 2023-06-09T04:34:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.952+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87c0740e0 con 0x7fe87c0ae630 2023-06-09T04:34:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:54.955+0000 7fe871ffb700 1 -- 172.21.15.92:0/783260888 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe878017070 con 0x7fe87c0ae630 2023-06-09T04:34:55.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.023+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe87c06e1d0 con 0x7fe87c0ae630 2023-06-09T04:34:55.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.024+0000 7fe871ffb700 1 -- 172.21.15.92:0/783260888 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe87801b570 con 0x7fe87c0ae630 2023-06-09T04:34:55.025 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:34:55.026 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:34:54.721798+0000","last_up_change":"0.000000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:34:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe87c0ae630 msgr2=0x7fe87c0c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe87c0ae630 0x7fe87c0c5d80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe87800c700 tx=0x7fe87800ca10 comp rx=0 tx=0).stop 2023-06-09T04:34:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 shutdown_connections 2023-06-09T04:34:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe87c0cd7e0 0x7fe87c06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe87c0ae630 0x7fe87c0c5d80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 --2- 172.21.15.92:0/783260888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c0b1360 0x7fe87c15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 >> 172.21.15.92:0/783260888 conn(0x7fe87c098c10 msgr2=0x7fe87c0b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:55.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 shutdown_connections 2023-06-09T04:34:55.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:34:55.025+0000 7fe88362a700 1 -- 172.21.15.92:0/783260888 wait complete. 2023-06-09T04:34:55.038 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-09T04:34:55.038 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-09T04:34:55.335 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:55.334+0000 7f0349711200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-09T04:34:55.356 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-06-09T04:34:55.355+0000 7f9897bc7640 -1 Falling back to public interface 2023-06-09T04:34:55.376 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-06-09T04:34:55.375+0000 7f48245c7640 -1 Falling back to public interface 2023-06-09T04:34:55.386 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-06-09T04:34:55.385+0000 7f9561f32640 -1 Falling back to public interface 2023-06-09T04:34:55.391 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-06-09T04:34:55.390+0000 7f02e6adf640 -1 Falling back to public interface 2023-06-09T04:34:55.392 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-06-09T04:34:55.390+0000 7f1e4b6c0640 -1 Falling back to public interface 2023-06-09T04:34:55.392 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-06-09T04:34:55.391+0000 7f6b163d0640 -1 Falling back to public interface 2023-06-09T04:34:55.396 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-06-09T04:34:55.395+0000 7fef6fe7a640 -1 Falling back to public interface 2023-06-09T04:34:55.417 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:55.415+0000 7f0349711200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-09T04:34:55.423 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-06-09T04:34:55.422+0000 7fa628783640 -1 Falling back to public interface 2023-06-09T04:34:55.426 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:55.425+0000 7f01cd798200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-09T04:34:55.504 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:55.503+0000 7f01cd798200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-09T04:34:55.556 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:55.555+0000 7f0349711200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-09T04:34:55.641 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:55.640+0000 7f01cd798200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-09T04:34:55.791 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:55.789+0000 7f0349711200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-09T04:34:55.844 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-09T04:34:55.864 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:55.863+0000 7f0349711200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-09T04:34:55.876 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:55.874+0000 7f01cd798200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-09T04:34:55.949 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:55.947+0000 7f01cd798200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-09T04:34:55.959 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.958+0000 7f3526056700 1 Processor -- start 2023-06-09T04:34:55.959 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.958+0000 7f3526056700 1 -- start start 2023-06-09T04:34:55.961 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.958+0000 7f3526056700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200b4df0 0x7f35200d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:55.961 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.958+0000 7f3526056700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35200d1140 0x7f35200b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:55.961 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.958+0000 7f3526056700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35200b2520 con 0x7f352015fd80 2023-06-09T04:34:55.961 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.958+0000 7f3526056700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200d1970 con 0x7f35200d1140 2023-06-09T04:34:55.961 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.958+0000 7f3526056700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35200d1c50 con 0x7f35200b4df0 2023-06-09T04:34:55.962 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f352015fd80 0x7f35200b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35200/0 (socket says 172.21.15.92:35200) 2023-06-09T04:34:55.962 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351f7fe700 1 -- 172.21.15.92:0/1526565219 learned_addr learned my addr 172.21.15.92:0/1526565219 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:34:55.962 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351ffff700 1 --2- 172.21.15.92:0/1526565219 >> v2:172.21.15.123:3300/0 conn(0x7f35200d1140 0x7f35200b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:55.962 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f3516ffd700 1 --2- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200b4df0 0x7f35200d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:55.962 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351effd700 1 -- 172.21.15.92:0/1526565219 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983378900 0 0) 0x7f3510001110 con 0x7f352015fd80 2023-06-09T04:34:55.962 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351effd700 1 -- 172.21.15.92:0/1526565219 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f350c0035b0 con 0x7f352015fd80 2023-06-09T04:34:55.962 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351ffff700 1 -- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200b4df0 msgr2=0x7f35200d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351ffff700 1 --2- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200b4df0 0x7f35200d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351ffff700 1 -- 172.21.15.92:0/1526565219 >> v1:172.21.15.92:6789/0 conn(0x7f352015fd80 legacy=0x7f35200b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351ffff700 1 -- 172.21.15.92:0/1526565219 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200d1f30 con 0x7f35200d1140 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f3516ffd700 1 --2- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200b4df0 0x7f35200d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351ffff700 1 --2- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35200d1140 0x7f35200b6350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3518009d90 tx=0x7f351800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11345c08c477490 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351effd700 1 -- 172.21.15.92:0/1526565219 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351800e070 con 0x7f35200d1140 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351effd700 1 -- 172.21.15.92:0/1526565219 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351800bb20 con 0x7f35200d1140 2023-06-09T04:34:55.963 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.959+0000 7f351effd700 1 -- 172.21.15.92:0/1526565219 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518012430 con 0x7f35200d1140 2023-06-09T04:34:55.966 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 -- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35200d1140 msgr2=0x7f35200b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:55.966 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 --2- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35200d1140 0x7f35200b6350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3518009d90 tx=0x7f351800b330 comp rx=0 tx=0).stop 2023-06-09T04:34:55.966 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 -- 172.21.15.92:0/1526565219 shutdown_connections 2023-06-09T04:34:55.966 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 --2- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35200d1140 0x7f35200b6350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.966 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 --2- 172.21.15.92:0/1526565219 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200b4df0 0x7f35200d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.966 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 -- 172.21.15.92:0/1526565219 >> 172.21.15.92:0/1526565219 conn(0x7f35200cf6e0 msgr2=0x7f35200cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:55.966 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 -- 172.21.15.92:0/1526565219 shutdown_connections 2023-06-09T04:34:55.967 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.960+0000 7f3526056700 1 -- 172.21.15.92:0/1526565219 wait complete. 2023-06-09T04:34:55.967 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.961+0000 7f3526056700 1 Processor -- start 2023-06-09T04:34:55.967 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3526056700 1 -- start start 2023-06-09T04:34:55.967 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3526056700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35200b4df0 0x7f35201502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:55.967 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3526056700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200d1140 0x7f35200cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:55.967 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3526056700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f352015fd80 0x7f35200c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:34:55.967 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3526056700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200c79b0 con 0x7f35200b4df0 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3526056700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35200c6d30 con 0x7f352015fd80 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3526056700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35200c7010 con 0x7f35200d1140 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35200b4df0 0x7f35201502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35200b4df0 0x7f35201502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36874/0 (socket says 172.21.15.92:36874) 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f351f7fe700 1 -- 172.21.15.92:0/3480200247 learned_addr learned my addr 172.21.15.92:0/3480200247 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f3516ffd700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200d1140 0x7f35200cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.962+0000 7f351ffff700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f352015fd80 0x7f35200c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:34:55.968 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3516ffd700 1 -- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f352015fd80 msgr2=0x7f35200c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:55.969 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3516ffd700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f352015fd80 0x7f35200c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.969 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3516ffd700 1 -- 172.21.15.92:0/3480200247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35200b4df0 msgr2=0x7f35201502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:55.969 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3516ffd700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35200b4df0 0x7f35201502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:55.969 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3516ffd700 1 -- 172.21.15.92:0/3480200247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080094a0 con 0x7f35200d1140 2023-06-09T04:34:55.969 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3516ffd700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200d1140 0x7f35200cc420 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f352008c110 tx=0x7f350800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:34:55.969 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f351cff9700 1 -- 172.21.15.92:0/3480200247 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508013680 con 0x7f35200d1140 2023-06-09T04:34:55.969 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f351cff9700 1 -- 172.21.15.92:0/3480200247 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35080137e0 con 0x7f35200d1140 2023-06-09T04:34:55.970 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3518009a70 con 0x7f35200d1140 2023-06-09T04:34:55.970 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f351cff9700 1 -- 172.21.15.92:0/3480200247 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508013b70 con 0x7f35200d1140 2023-06-09T04:34:55.970 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35200c75d0 con 0x7f35200d1140 2023-06-09T04:34:55.970 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f351cff9700 1 -- 172.21.15.92:0/3480200247 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3508005a20 con 0x7f35200d1140 2023-06-09T04:34:55.970 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.963+0000 7f351cff9700 1 -- 172.21.15.92:0/3480200247 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3508005cd0 con 0x7f35200d1140 2023-06-09T04:34:55.970 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.964+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3520066f60 con 0x7f35200d1140 2023-06-09T04:34:55.971 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:55.966+0000 7f351cff9700 1 -- 172.21.15.92:0/3480200247 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3508021060 con 0x7f35200d1140 2023-06-09T04:34:56.037 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.036+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35200c7af0 con 0x7f35200d1140 2023-06-09T04:34:56.038 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.036+0000 7f351cff9700 1 -- 172.21.15.92:0/3480200247 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f35200c7af0 con 0x7f35200d1140 2023-06-09T04:34:56.038 INFO:teuthology.misc.health.smithi092.stdout: 2023-06-09T04:34:56.038 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:34:54.721798+0000","last_up_change":"0.000000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:34:56.039 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.037+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200d1140 msgr2=0x7f35200cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:34:56.040 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.037+0000 7f3526056700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200d1140 0x7f35200cc420 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f352008c110 tx=0x7f350800ec60 comp rx=0 tx=0).stop 2023-06-09T04:34:56.040 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.037+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 shutdown_connections 2023-06-09T04:34:56.040 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.037+0000 7f3526056700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f352015fd80 0x7f35200c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:56.040 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.037+0000 7f3526056700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35200b4df0 0x7f35201502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:56.040 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.037+0000 7f3526056700 1 --2- 172.21.15.92:0/3480200247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35200d1140 0x7f35200cc420 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:34:56.040 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.037+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 >> 172.21.15.92:0/3480200247 conn(0x7f35200cf6e0 msgr2=0x7f352006b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:34:56.040 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.038+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 shutdown_connections 2023-06-09T04:34:56.041 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:34:56.038+0000 7f3526056700 1 -- 172.21.15.92:0/3480200247 wait complete. 2023-06-09T04:34:56.050 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-09T04:34:56.142 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:56.141+0000 7f0349711200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-09T04:34:56.219 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:56.217+0000 7f01cd798200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-09T04:34:56.360 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:56.359+0000 7f0349711200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-09T04:34:56.437 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:56.435+0000 7f01cd798200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-09T04:34:56.516 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:56.514+0000 7f0349711200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-09T04:34:56.593 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:56.592+0000 7f01cd798200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-09T04:34:56.629 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:56.628+0000 7f0349711200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-09T04:34:56.707 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:56.706+0000 7f01cd798200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-09T04:34:56.948 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:56.947+0000 7f0349711200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-09T04:34:57.027 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:57.026+0000 7f01cd798200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-09T04:34:57.048 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:57.046+0000 7f0349711200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-09T04:34:57.128 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:57.127+0000 7f01cd798200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-09T04:34:57.233 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:57.232+0000 7f0349711200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-09T04:34:57.309 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:57.308+0000 7f01cd798200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-09T04:34:57.822 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:57.821+0000 7f0349711200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-09T04:34:57.905 INFO:tasks.ceph.mgr.x.smithi123.stderr:2023-06-09T04:34:57.904+0000 7f01cd798200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-09T04:34:58.846 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:58.844+0000 7f030193d700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-09T04:34:58.848 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-06-09T04:34:58.847+0000 7f030193d700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-09T04:34:58.898 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-06-09T04:34:58.897+0000 7f9897bc7640 -1 osd.1 0 log_to_monitors true 2023-06-09T04:34:58.913 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-06-09T04:34:58.912+0000 7f48245c7640 -1 osd.0 0 log_to_monitors true 2023-06-09T04:34:58.947 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-06-09T04:34:58.946+0000 7fef6fe7a640 -1 osd.7 0 log_to_monitors true 2023-06-09T04:34:58.949 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-06-09T04:34:58.948+0000 7f1e4b6c0640 -1 osd.2 0 log_to_monitors true 2023-06-09T04:34:58.949 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-06-09T04:34:58.948+0000 7f9561f32640 -1 osd.3 0 log_to_monitors true 2023-06-09T04:34:58.963 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-06-09T04:34:58.961+0000 7f6b163d0640 -1 osd.5 0 log_to_monitors true 2023-06-09T04:34:58.963 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-06-09T04:34:58.962+0000 7fa628783640 -1 osd.6 0 log_to_monitors true 2023-06-09T04:34:58.963 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-06-09T04:34:58.962+0000 7f02e6adf640 -1 osd.4 0 log_to_monitors true 2023-06-09T04:35:02.855 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:02.874 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-06-09T04:35:02.873+0000 7f9555ae9700 -1 osd.3 0 waiting for initial osdmap 2023-06-09T04:35:02.875 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-06-09T04:35:02.873+0000 7f481817e700 -1 osd.0 0 waiting for initial osdmap 2023-06-09T04:35:02.880 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-06-09T04:35:02.879+0000 7f02da696700 -1 osd.4 0 waiting for initial osdmap 2023-06-09T04:35:02.880 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-06-09T04:35:02.878+0000 7fef63a31700 -1 osd.7 0 waiting for initial osdmap 2023-06-09T04:35:02.888 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-06-09T04:35:02.887+0000 7f4810559700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.888 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-06-09T04:35:02.887+0000 7f954dec4700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.892 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-06-09T04:35:02.891+0000 7f02d2a71700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.894 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-06-09T04:35:02.893+0000 7f1e3f277700 -1 osd.2 0 waiting for initial osdmap 2023-06-09T04:35:02.898 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-06-09T04:35:02.897+0000 7f988b77e700 -1 osd.1 0 waiting for initial osdmap 2023-06-09T04:35:02.899 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-06-09T04:35:02.898+0000 7fa61c33a700 -1 osd.6 0 waiting for initial osdmap 2023-06-09T04:35:02.900 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-06-09T04:35:02.899+0000 7f6b09f87700 -1 osd.5 0 waiting for initial osdmap 2023-06-09T04:35:02.901 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-06-09T04:35:02.900+0000 7fef5be0c700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.907 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-06-09T04:35:02.906+0000 7f1e37652700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.912 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-06-09T04:35:02.911+0000 7fa614715700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.912 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-06-09T04:35:02.911+0000 7f9883b59700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.916 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-06-09T04:35:02.915+0000 7f6b02362700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T04:35:02.982 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.980+0000 7f1871d60700 1 Processor -- start 2023-06-09T04:35:02.982 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f1871d60700 1 -- start start 2023-06-09T04:35:02.984 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f1871d60700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c06ea70 0x7f186c0b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:02.984 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f1871d60700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f186c0b1a10 0x7f186c0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:02.984 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f1871d60700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c0b1e60 con 0x7f186c0aef80 2023-06-09T04:35:02.984 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f1871d60700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c0af350 con 0x7f186c0b1a10 2023-06-09T04:35:02.984 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f1871d60700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1598c0 con 0x7f186c06ea70 2023-06-09T04:35:02.985 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f186bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f186c0b1a10 0x7f186c0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:02.985 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.981+0000 7f186bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f186c0b1a10 0x7f186c0cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36982/0 (socket says 172.21.15.92:36982) 2023-06-09T04:35:02.985 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186bfff700 1 -- 172.21.15.92:0/464966184 learned_addr learned my addr 172.21.15.92:0/464966184 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:02.985 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186affd700 1 --2- 172.21.15.92:0/464966184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c06ea70 0x7f186c0b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:02.985 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186bfff700 1 -- 172.21.15.92:0/464966184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c06ea70 msgr2=0x7f186c0b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:02.985 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186bfff700 1 --2- 172.21.15.92:0/464966184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c06ea70 0x7f186c0b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:02.985 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186bfff700 1 -- 172.21.15.92:0/464966184 >> v1:172.21.15.123:6789/0 conn(0x7f186c0aef80 legacy=0x7f186c0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186bfff700 1 -- 172.21.15.92:0/464966184 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c06eee0 con 0x7f186c0b1a10 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186bfff700 1 --2- 172.21.15.92:0/464966184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c0b1a10 0x7f186c0cab90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e80 tx=0x7f186400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8616873e49ba2701 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186a7fc700 1 -- 172.21.15.92:0/464966184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400e070 con 0x7f186c0b1a10 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186a7fc700 1 -- 172.21.15.92:0/464966184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186400b980 con 0x7f186c0b1a10 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.982+0000 7f186a7fc700 1 -- 172.21.15.92:0/464966184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400bd30 con 0x7f186c0b1a10 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 -- 172.21.15.92:0/464966184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c0b1a10 msgr2=0x7f186c0cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 --2- 172.21.15.92:0/464966184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c0b1a10 0x7f186c0cab90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e80 tx=0x7f186400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:02.986 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 -- 172.21.15.92:0/464966184 shutdown_connections 2023-06-09T04:35:02.987 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 --2- 172.21.15.92:0/464966184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c0b1a10 0x7f186c0cab90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:02.987 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 --2- 172.21.15.92:0/464966184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c06ea70 0x7f186c0b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:02.987 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 -- 172.21.15.92:0/464966184 >> 172.21.15.92:0/464966184 conn(0x7f186c098c80 msgr2=0x7f186c0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:02.987 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 -- 172.21.15.92:0/464966184 shutdown_connections 2023-06-09T04:35:02.987 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.983+0000 7f1871d60700 1 -- 172.21.15.92:0/464966184 wait complete. 2023-06-09T04:35:02.989 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.984+0000 7f1871d60700 1 Processor -- start 2023-06-09T04:35:02.989 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.984+0000 7f1871d60700 1 -- start start 2023-06-09T04:35:02.990 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c06ea70 0x7f186c153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:02.990 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c0aef80 0x7f186c1576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:02.990 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f186c0b1a10 0x7f186c157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:02.990 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1e1690 con 0x7f186c06ea70 2023-06-09T04:35:02.990 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1df040 con 0x7f186c0b1a10 2023-06-09T04:35:02.990 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1ded10 con 0x7f186c0aef80 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c06ea70 0x7f186c153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c06ea70 0x7f186c153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36984/0 (socket says 172.21.15.92:36984) 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 -- 172.21.15.92:0/1107048840 learned_addr learned my addr 172.21.15.92:0/1107048840 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186affd700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c0aef80 0x7f186c1576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186bfff700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f186c0b1a10 0x7f186c157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 -- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c0aef80 msgr2=0x7f186c1576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c0aef80 0x7f186c1576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:02.991 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 -- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f186c0b1a10 msgr2=0x7f186c157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:02.992 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f186c0b1a10 0x7f186c157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:02.992 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 -- 172.21.15.92:0/1107048840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1864009b30 con 0x7f186c06ea70 2023-06-09T04:35:02.992 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f186b7fe700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c06ea70 0x7f186c153ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c4d0 tx=0x7f185c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:02.992 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1853fff700 1 -- 172.21.15.92:0/1107048840 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c0052d0 con 0x7f186c06ea70 2023-06-09T04:35:02.992 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1853fff700 1 -- 172.21.15.92:0/1107048840 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c005430 con 0x7f186c06ea70 2023-06-09T04:35:02.992 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c1e10d0 con 0x7f186c06ea70 2023-06-09T04:35:02.992 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1853fff700 1 -- 172.21.15.92:0/1107048840 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c0038f0 con 0x7f186c06ea70 2023-06-09T04:35:02.993 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.985+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c0c6cf0 con 0x7f186c06ea70 2023-06-09T04:35:02.993 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.986+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c1deb70 con 0x7f186c06ea70 2023-06-09T04:35:02.993 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.987+0000 7f1853fff700 1 -- 172.21.15.92:0/1107048840 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f185c003a50 con 0x7f186c06ea70 2023-06-09T04:35:02.993 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.987+0000 7f1853fff700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f185806ac50 0x7f185806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:02.993 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.987+0000 7f1853fff700 1 -- 172.21.15.92:0/1107048840 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f185c087770 con 0x7f186c06ea70 2023-06-09T04:35:02.993 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.987+0000 7f186affd700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f185806ac50 0x7f185806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:02.993 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.987+0000 7f186affd700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f185806ac50 0x7f185806d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f186c1e4d50 tx=0x7f1860008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:02.994 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:02.992+0000 7f1853fff700 1 -- 172.21.15.92:0/1107048840 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f186c1deb70 con 0x7f186c06ea70 2023-06-09T04:35:03.181 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.179+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f186c0b1e60 con 0x7f186c06ea70 2023-06-09T04:35:03.181 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.180+0000 7f1853fff700 1 -- 172.21.15.92:0/1107048840 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f186c0b1e60 con 0x7f186c06ea70 2023-06-09T04:35:03.182 INFO:teuthology.misc.health.smithi092.stdout: 2023-06-09T04:35:03.182 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":12,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:00.841923+0000","last_up_change":"0.000000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:03.183 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f185806ac50 msgr2=0x7f185806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:03.184 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f185806ac50 0x7f185806d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f186c1e4d50 tx=0x7f1860008040 comp rx=0 tx=0).stop 2023-06-09T04:35:03.184 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c06ea70 msgr2=0x7f186c153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:03.184 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c06ea70 0x7f186c153ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c4d0 tx=0x7f185c00c7e0 comp rx=0 tx=0).stop 2023-06-09T04:35:03.184 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 shutdown_connections 2023-06-09T04:35:03.184 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f185806ac50 0x7f185806d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:03.184 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f186c0b1a10 0x7f186c157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:03.184 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f186c06ea70 0x7f186c153ae0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:03.185 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 --2- 172.21.15.92:0/1107048840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f186c0aef80 0x7f186c1576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:03.185 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 >> 172.21.15.92:0/1107048840 conn(0x7f186c098c80 msgr2=0x7f186c0ac650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:03.185 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 shutdown_connections 2023-06-09T04:35:03.185 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:03.182+0000 7f1871d60700 1 -- 172.21.15.92:0/1107048840 wait complete. 2023-06-09T04:35:03.194 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-09T04:35:09.999 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:10.125 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.124+0000 7f9a117bc700 1 Processor -- start 2023-06-09T04:35:10.125 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.124+0000 7f9a117bc700 1 -- start start 2023-06-09T04:35:10.126 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.124+0000 7f9a117bc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 0x7f9a0c06cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.126 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a117bc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0b0890 con 0x7f9a0c06da30 2023-06-09T04:35:10.126 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a117bc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c158cb0 con 0x7f9a0c070a20 2023-06-09T04:35:10.126 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a117bc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cb440 con 0x7f9a0c1588e0 2023-06-09T04:35:10.127 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a0a7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9a0c06da30 0x7f9a0c070310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35550/0 (socket says 172.21.15.92:35550) 2023-06-09T04:35:10.127 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a0a7fc700 1 -- 172.21.15.92:0/2950902273 learned_addr learned my addr 172.21.15.92:0/2950902273 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:10.127 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a0affd700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 0x7f9a0c06cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.127 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056153633 0 0) 0x7f9a00001110 con 0x7f9a0c06da30 2023-06-09T04:35:10.127 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f40035b0 con 0x7f9a0c06da30 2023-06-09T04:35:10.129 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361986123 0 0) 0x7f99f40035b0 con 0x7f9a0c06da30 2023-06-09T04:35:10.129 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 msgr2=0x7f9a0c06cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 0x7f9a0c06cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 >> v1:172.21.15.123:6789/0 conn(0x7f9a0c070a20 legacy=0x7f9a0c0adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0af7f0 con 0x7f9a0c06da30 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f9a00001800 con 0x7f9a0c06da30 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 >> v1:172.21.15.92:6789/0 conn(0x7f9a0c06da30 legacy=0x7f9a0c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f4003a30 0x7f99f4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 0x7f99f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.130 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99f4004d90 0x7f99f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.131 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cb440 con 0x7f99f4003a30 2023-06-09T04:35:10.131 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f400bac0 con 0x7f99f4004d90 2023-06-09T04:35:10.131 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f400bc50 con 0x7f9a0c1588e0 2023-06-09T04:35:10.131 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.125+0000 7f9a0affd700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f4003a30 0x7f99f4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.131 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0a7fc700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 0x7f99f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.131 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0b7fe700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99f4004d90 0x7f99f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.131 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0affd700 1 -- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 msgr2=0x7f99f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0affd700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9a0c1588e0 0x7f99f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0affd700 1 -- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99f4004d90 msgr2=0x7f99f4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0affd700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99f4004d90 0x7f99f4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0affd700 1 -- 172.21.15.92:0/2950902273 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40035b0 con 0x7f99f4003a30 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a0affd700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f4003a30 0x7f99f4003e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b60 tx=0x7f99f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c35067d09bd6af server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00e6c0 con 0x7f99f4003a30 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00e820 con 0x7f99f4003a30 2023-06-09T04:35:10.132 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.126+0000 7f9a09ffb700 1 -- 172.21.15.92:0/2950902273 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00eb90 con 0x7f99f4003a30 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.127+0000 7f9a117bc700 1 -- 172.21.15.92:0/2950902273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f4003a30 msgr2=0x7f99f4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.127+0000 7f9a117bc700 1 --2- 172.21.15.92:0/2950902273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99f4003a30 0x7f99f4003e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b60 tx=0x7f99f4005880 comp rx=0 tx=0).stop 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.127+0000 7f9a0b7fe700 1 -- 172.21.15.92:0/2950902273 reap_dead start 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.127+0000 7f9a117bc700 1 -- 172.21.15.92:0/2950902273 shutdown_connections 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.127+0000 7f9a117bc700 1 -- 172.21.15.92:0/2950902273 >> 172.21.15.92:0/2950902273 conn(0x7f9a0c065ae0 msgr2=0x7f9a0c065eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.127+0000 7f9a117bc700 1 -- 172.21.15.92:0/2950902273 shutdown_connections 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.127+0000 7f9a117bc700 1 -- 172.21.15.92:0/2950902273 wait complete. 2023-06-09T04:35:10.133 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.128+0000 7f9a117bc700 1 Processor -- start 2023-06-09T04:35:10.134 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.128+0000 7f9a117bc700 1 -- start start 2023-06-09T04:35:10.135 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a117bc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99f4003a30 0x7f9a0c0ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.135 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a117bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c15a370 0x7f9a0c15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.136 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a117bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0c159210 0x7f9a0c159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.136 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a117bc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cc3b0 con 0x7f9a0c159210 2023-06-09T04:35:10.136 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a117bc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c1424b0 con 0x7f9a0c15a370 2023-06-09T04:35:10.136 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a117bc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c15ad20 con 0x7f99f4003a30 2023-06-09T04:35:10.136 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0c159210 0x7f9a0c159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.136 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0c159210 0x7f9a0c159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37232/0 (socket says 172.21.15.92:37232) 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 -- 172.21.15.92:0/1050989100 learned_addr learned my addr 172.21.15.92:0/1050989100 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0affd700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99f4003a30 0x7f9a0c0ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0a7fc700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c15a370 0x7f9a0c15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 -- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99f4003a30 msgr2=0x7f9a0c0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99f4003a30 0x7f9a0c0ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 -- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c15a370 msgr2=0x7f9a0c15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c15a370 0x7f9a0c15a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.137 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 -- 172.21.15.92:0/1050989100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0cd630 con 0x7f9a0c159210 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f9a0b7fe700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0c159210 0x7f9a0c159680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0cb1b0 tx=0x7f9a0400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.129+0000 7f99f37fe700 1 -- 172.21.15.92:0/1050989100 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04003c10 con 0x7f9a0c159210 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.130+0000 7f99f37fe700 1 -- 172.21.15.92:0/1050989100 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04003d90 con 0x7f9a0c159210 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.130+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c0cc590 con 0x7f9a0c159210 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.130+0000 7f99f37fe700 1 -- 172.21.15.92:0/1050989100 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0400db70 con 0x7f9a0c159210 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.130+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c0ccb50 con 0x7f9a0c159210 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.131+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c04e090 con 0x7f9a0c159210 2023-06-09T04:35:10.138 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.131+0000 7f99f37fe700 1 -- 172.21.15.92:0/1050989100 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a0400dcd0 con 0x7f9a0c159210 2023-06-09T04:35:10.139 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.131+0000 7f99f37fe700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f99f806ace0 0x7f99f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.139 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.131+0000 7f99f37fe700 1 -- 172.21.15.92:0/1050989100 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9a040891e0 con 0x7f9a0c159210 2023-06-09T04:35:10.139 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.131+0000 7f9a0affd700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f99f806ace0 0x7f99f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.139 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.132+0000 7f9a0affd700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f99f806ace0 0x7f99f806d190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0090b0 tx=0x7f99fc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:10.139 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.136+0000 7f99f37fe700 1 -- 172.21.15.92:0/1050989100 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a040547d0 con 0x7f9a0c159210 2023-06-09T04:35:10.324 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.323+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a0c070560 con 0x7f9a0c159210 2023-06-09T04:35:10.325 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.324+0000 7f99f37fe700 1 -- 172.21.15.92:0/1050989100 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f9a04057df0 con 0x7f9a0c159210 2023-06-09T04:35:10.325 INFO:teuthology.misc.health.smithi092.stdout: 2023-06-09T04:35:10.325 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":17,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:07.920344+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:10.327 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f99f806ace0 msgr2=0x7f99f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.328 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f99f806ace0 0x7f99f806d190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0090b0 tx=0x7f99fc008030 comp rx=0 tx=0).stop 2023-06-09T04:35:10.328 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0c159210 msgr2=0x7f9a0c159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.328 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0c159210 0x7f9a0c159680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0cb1b0 tx=0x7f9a0400d430 comp rx=0 tx=0).stop 2023-06-09T04:35:10.328 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 shutdown_connections 2023-06-09T04:35:10.328 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f99f806ace0 0x7f99f806d190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.328 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a0c159210 0x7f9a0c159680 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.329 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99f4003a30 0x7f9a0c0ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.329 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 --2- 172.21.15.92:0/1050989100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c15a370 0x7f9a0c15a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.329 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 >> 172.21.15.92:0/1050989100 conn(0x7f9a0c065ae0 msgr2=0x7f9a0c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:10.329 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 shutdown_connections 2023-06-09T04:35:10.329 INFO:teuthology.misc.health.smithi092.stderr:2023-06-09T04:35:10.326+0000 7f9a117bc700 1 -- 172.21.15.92:0/1050989100 wait complete. 2023-06-09T04:35:10.338 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-09T04:35:10.339 INFO:tasks.ceph:Creating RBD pool 2023-06-09T04:35:10.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-09T04:35:10.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.456+0000 7f10519a0700 1 Processor -- start 2023-06-09T04:35:10.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.456+0000 7f10519a0700 1 -- start start 2023-06-09T04:35:10.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f10519a0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f104c070a30 0x7f104c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f10519a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f104c0b16e0 0x7f104c1600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f10519a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f10519a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104c0cd550 con 0x7f104c0b16e0 2023-06-09T04:35:10.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f10519a0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104c06b290 con 0x7f104c070a30 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f10519a0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0ccf90 con 0x7f104c070e00 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f104a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f104c0b16e0 0x7f104c1600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f104a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f104c0b16e0 0x7f104c1600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37236/0 (socket says 172.21.15.92:37236) 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.457+0000 7f104a7fc700 1 -- 172.21.15.92:0/118413521 learned_addr learned my addr 172.21.15.92:0/118413521 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104b7fe700 1 --2- 172.21.15.92:0/118413521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104a7fc700 1 -- 172.21.15.92:0/118413521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 msgr2=0x7f104c15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104affd700 1 --2- 172.21.15.92:0/118413521 >> v2:172.21.15.123:3300/0 conn(0x7f104c070a30 0x7f104c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104a7fc700 1 --2- 172.21.15.92:0/118413521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c15ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104a7fc700 1 -- 172.21.15.92:0/118413521 >> v2:172.21.15.123:3300/0 conn(0x7f104c070a30 msgr2=0x7f104c06d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104a7fc700 1 --2- 172.21.15.92:0/118413521 >> v2:172.21.15.123:3300/0 conn(0x7f104c070a30 0x7f104c06d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104a7fc700 1 -- 172.21.15.92:0/118413521 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c0cd270 con 0x7f104c0b16e0 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f104a7fc700 1 --2- 172.21.15.92:0/118413521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0b16e0 0x7f104c1600b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f104c06e200 tx=0x7f103c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e666685701026c9e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f1049ffb700 1 -- 172.21.15.92:0/118413521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00b870 con 0x7f104c0b16e0 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f1049ffb700 1 -- 172.21.15.92:0/118413521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c00b9d0 con 0x7f104c0b16e0 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f1049ffb700 1 -- 172.21.15.92:0/118413521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00bd60 con 0x7f104c0b16e0 2023-06-09T04:35:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 -- 172.21.15.92:0/118413521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0b16e0 msgr2=0x7f104c1600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 --2- 172.21.15.92:0/118413521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0b16e0 0x7f104c1600b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f104c06e200 tx=0x7f103c009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 -- 172.21.15.92:0/118413521 shutdown_connections 2023-06-09T04:35:10.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 --2- 172.21.15.92:0/118413521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 --2- 172.21.15.92:0/118413521 >> v2:172.21.15.123:3300/0 conn(0x7f104c070a30 0x7f104c06d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 --2- 172.21.15.92:0/118413521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c0b16e0 0x7f104c1600b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 -- 172.21.15.92:0/118413521 >> 172.21.15.92:0/118413521 conn(0x7f104c0cfc40 msgr2=0x7f104c0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:10.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.458+0000 7f10519a0700 1 -- 172.21.15.92:0/118413521 shutdown_connections 2023-06-09T04:35:10.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.459+0000 7f10519a0700 1 -- 172.21.15.92:0/118413521 wait complete. 2023-06-09T04:35:10.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.460+0000 7f10519a0700 1 Processor -- start 2023-06-09T04:35:10.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.460+0000 7f10519a0700 1 -- start start 2023-06-09T04:35:10.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c070a30 0x7f104c1e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c0b16e0 0x7f104c0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0adfe0 con 0x7f104c070a30 2023-06-09T04:35:10.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0ad3d0 con 0x7f104c0b16e0 2023-06-09T04:35:10.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0ad6b0 con 0x7f104c070e00 2023-06-09T04:35:10.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c070a30 0x7f104c1e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c070a30 0x7f104c1e5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37242/0 (socket says 172.21.15.92:37242) 2023-06-09T04:35:10.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 -- 172.21.15.92:0/282179599 learned_addr learned my addr 172.21.15.92:0/282179599 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:10.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104a7fc700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104b7fe700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c0b16e0 0x7f104c0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 -- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 msgr2=0x7f104c0b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c0b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 -- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c0b16e0 msgr2=0x7f104c0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c0b16e0 0x7f104c0b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 -- 172.21.15.92:0/282179599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c009550 con 0x7f104c070a30 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104b7fe700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c0b16e0 0x7f104c0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f104affd700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c070a30 0x7f104c1e5f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f104000c7f0 tx=0x7f104000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f1033fff700 1 -- 172.21.15.92:0/282179599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040011800 con 0x7f104c070a30 2023-06-09T04:35:10.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f1033fff700 1 -- 172.21.15.92:0/282179599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1040011960 con 0x7f104c070a30 2023-06-09T04:35:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c0ad990 con 0x7f104c070a30 2023-06-09T04:35:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f1033fff700 1 -- 172.21.15.92:0/282179599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040011d10 con 0x7f104c070a30 2023-06-09T04:35:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.461+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c154520 con 0x7f104c070a30 2023-06-09T04:35:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.462+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104c0ae2c0 con 0x7f104c070a30 2023-06-09T04:35:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.463+0000 7f1033fff700 1 -- 172.21.15.92:0/282179599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1040018360 con 0x7f104c070a30 2023-06-09T04:35:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.463+0000 7f1033fff700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f103806ace0 0x7f103806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:10.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.463+0000 7f1033fff700 1 -- 172.21.15.92:0/282179599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1040088a90 con 0x7f104c070a30 2023-06-09T04:35:10.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.463+0000 7f104a7fc700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f103806ace0 0x7f103806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:10.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.463+0000 7f104a7fc700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f103806ace0 0x7f103806d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f104c0b0930 tx=0x7f103c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:10.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.467+0000 7f1033fff700 1 -- 172.21.15.92:0/282179599 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1040054080 con 0x7f104c070a30 2023-06-09T04:35:10.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.655+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f104c04e8e0 con 0x7f104c070a30 2023-06-09T04:35:10.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.988+0000 7f1033fff700 1 -- 172.21.15.92:0/282179599 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f10400576a0 con 0x7f104c070a30 2023-06-09T04:35:10.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f103806ace0 msgr2=0x7f103806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f103806ace0 0x7f103806d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f104c0b0930 tx=0x7f103c008030 comp rx=0 tx=0).stop 2023-06-09T04:35:10.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c070a30 msgr2=0x7f104c1e5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c070a30 0x7f104c1e5f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f104000c7f0 tx=0x7f104000cbb0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 shutdown_connections 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f103806ace0 0x7f103806d190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c0b16e0 0x7f104c0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f104c070a30 0x7f104c1e5f10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 --2- 172.21.15.92:0/282179599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f104c070e00 0x7f104c0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 >> 172.21.15.92:0/282179599 conn(0x7f104c0cfc40 msgr2=0x7f104c15b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:10.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 shutdown_connections 2023-06-09T04:35:10.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:10.992+0000 7f10519a0700 1 -- 172.21.15.92:0/282179599 wait complete. 2023-06-09T04:35:10.995 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2023-06-09T04:35:11.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-09T04:35:11.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.126+0000 7f85be9ea700 1 Processor -- start 2023-06-09T04:35:11.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.126+0000 7f85be9ea700 1 -- start start 2023-06-09T04:35:11.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85be9ea700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f85b80aed50 0x7f85b80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:11.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85be9ea700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b815d470 0x7f85b815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:11.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85be9ea700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b80ce190 con 0x7f85b80ce3d0 2023-06-09T04:35:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85be9ea700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b8072350 con 0x7f85b80aed50 2023-06-09T04:35:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85be9ea700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8071ab0 con 0x7f85b815d470 2023-06-09T04:35:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85bcf87700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b815d470 0x7f85b815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85b77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f85b80aed50 0x7f85b80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85bcf87700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b815d470 0x7f85b815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46162/0 (socket says 172.21.15.92:46162) 2023-06-09T04:35:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.127+0000 7f85bcf87700 1 -- 172.21.15.92:0/472346137 learned_addr learned my addr 172.21.15.92:0/472346137 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b77fe700 1 -- 172.21.15.92:0/472346137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b815d470 msgr2=0x7f85b815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b77fe700 1 --2- 172.21.15.92:0/472346137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b815d470 0x7f85b815d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b77fe700 1 -- 172.21.15.92:0/472346137 >> v1:172.21.15.123:6789/0 conn(0x7f85b80ce3d0 legacy=0x7f85b80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b77fe700 1 -- 172.21.15.92:0/472346137 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8071d90 con 0x7f85b80aed50 2023-06-09T04:35:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b77fe700 1 --2- 172.21.15.92:0/472346137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b80aed50 0x7f85b80713b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009840 tx=0x7f85a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618a5877fd44a6fc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b6ffd700 1 -- 172.21.15.92:0/472346137 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8011830 con 0x7f85b80aed50 2023-06-09T04:35:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b6ffd700 1 -- 172.21.15.92:0/472346137 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a8011990 con 0x7f85b80aed50 2023-06-09T04:35:11.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.128+0000 7f85b6ffd700 1 -- 172.21.15.92:0/472346137 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8011d20 con 0x7f85b80aed50 2023-06-09T04:35:11.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 -- 172.21.15.92:0/472346137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b80aed50 msgr2=0x7f85b80713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:11.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 --2- 172.21.15.92:0/472346137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b80aed50 0x7f85b80713b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009840 tx=0x7f85a8009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:11.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 -- 172.21.15.92:0/472346137 shutdown_connections 2023-06-09T04:35:11.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 --2- 172.21.15.92:0/472346137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b815d470 0x7f85b815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:11.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 --2- 172.21.15.92:0/472346137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b80aed50 0x7f85b80713b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:11.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 -- 172.21.15.92:0/472346137 >> 172.21.15.92:0/472346137 conn(0x7f85b80cc380 msgr2=0x7f85b80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 -- 172.21.15.92:0/472346137 shutdown_connections 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.129+0000 7f85be9ea700 1 -- 172.21.15.92:0/472346137 wait complete. 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.130+0000 7f85be9ea700 1 Processor -- start 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.130+0000 7f85be9ea700 1 -- start start 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85be9ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b80aed50 0x7f85b814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85be9ea700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b80ce3d0 0x7f85b80c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85be9ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b815d470 0x7f85b80c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:11.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85be9ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80c4870 con 0x7f85b80aed50 2023-06-09T04:35:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85be9ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b814f010 con 0x7f85b815d470 2023-06-09T04:35:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85be9ea700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80c5240 con 0x7f85b80ce3d0 2023-06-09T04:35:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b80aed50 0x7f85b814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b80aed50 0x7f85b814ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37256/0 (socket says 172.21.15.92:37256) 2023-06-09T04:35:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 -- 172.21.15.92:0/3273309815 learned_addr learned my addr 172.21.15.92:0/3273309815 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b77fe700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b80ce3d0 0x7f85b80c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:11.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85bcf87700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b815d470 0x7f85b80c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 -- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b80ce3d0 msgr2=0x7f85b80c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b80ce3d0 0x7f85b80c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 -- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b815d470 msgr2=0x7f85b80c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b815d470 0x7f85b80c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 -- 172.21.15.92:0/3273309815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a80094a0 con 0x7f85b80aed50 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85bcf87700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b815d470 0x7f85b80c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b77fe700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b80ce3d0 0x7f85b80c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:11.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b7fff700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b80aed50 0x7f85b814ead0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00c4e0 tx=0x7f85ac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.131+0000 7f85b4ff9700 1 -- 172.21.15.92:0/3273309815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0052d0 con 0x7f85b80aed50 2023-06-09T04:35:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.132+0000 7f85b4ff9700 1 -- 172.21.15.92:0/3273309815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac005430 con 0x7f85b80aed50 2023-06-09T04:35:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.132+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b80c3c60 con 0x7f85b80aed50 2023-06-09T04:35:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.132+0000 7f85b4ff9700 1 -- 172.21.15.92:0/3273309815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0038c0 con 0x7f85b80aed50 2023-06-09T04:35:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.132+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b80c4220 con 0x7f85b80aed50 2023-06-09T04:35:11.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.132+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b8072050 con 0x7f85b80aed50 2023-06-09T04:35:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.133+0000 7f85b4ff9700 1 -- 172.21.15.92:0/3273309815 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85ac003a20 con 0x7f85b80aed50 2023-06-09T04:35:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.133+0000 7f85b4ff9700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f85a406ad10 0x7f85a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.134+0000 7f85b4ff9700 1 -- 172.21.15.92:0/3273309815 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f85ac088870 con 0x7f85b80aed50 2023-06-09T04:35:11.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.134+0000 7f85b77fe700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f85a406ad10 0x7f85a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.134+0000 7f85b77fe700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f85a406ad10 0x7f85a406d1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:11.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.137+0000 7f85b4ff9700 1 -- 172.21.15.92:0/3273309815 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85ac053d60 con 0x7f85b80aed50 2023-06-09T04:35:11.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.323+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f85b8072050 con 0x7f85b80aed50 2023-06-09T04:35:11.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.996+0000 7f85b4ff9700 1 -- 172.21.15.92:0/3273309815 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f85b8072050 con 0x7f85b80aed50 2023-06-09T04:35:12.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f85a406ad10 msgr2=0x7f85a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f85a406ad10 0x7f85a406d1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a801c040 comp rx=0 tx=0).stop 2023-06-09T04:35:12.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b80aed50 msgr2=0x7f85b814ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b80aed50 0x7f85b814ead0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00c4e0 tx=0x7f85ac00c8a0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 shutdown_connections 2023-06-09T04:35:12.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f85a406ad10 0x7f85a406d1c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85b815d470 0x7f85b80c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85b80aed50 0x7f85b814ead0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 --2- 172.21.15.92:0/3273309815 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f85b80ce3d0 0x7f85b80c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.998+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 >> 172.21.15.92:0/3273309815 conn(0x7f85b80cc380 msgr2=0x7f85b80cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:12.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.999+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 shutdown_connections 2023-06-09T04:35:12.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:11.999+0000 7f85be9ea700 1 -- 172.21.15.92:0/3273309815 wait complete. 2023-06-09T04:35:12.002 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-09T04:35:12.014 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-09T04:35:12.014 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-09T04:35:12.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-09T04:35:12.016 INFO:tasks.ceph.mds.a:Started 2023-06-09T04:35:12.017 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-09T04:35:12.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-09T04:35:12.019 INFO:tasks.ceph.mds.c:Started 2023-06-09T04:35:12.019 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T04:35:12.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T04:35:12.021 INFO:tasks.ceph.mds.b:Started 2023-06-09T04:35:12.022 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T04:35:12.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T04:35:12.024 INFO:tasks.ceph.mds.d:Started 2023-06-09T04:35:12.024 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-09T04:35:12.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:12.099 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-06-09T04:35:12.102 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-06-09T04:35:12.109 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-06-09T04:35:12.109 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-06-09T04:35:12.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.156+0000 7f5dc392d700 1 Processor -- start 2023-06-09T04:35:12.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.156+0000 7f5dc392d700 1 -- start start 2023-06-09T04:35:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc392d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 0x7f5dbc0b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc392d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5dbc0ae630 0x7f5dbc0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc392d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dbc0adde0 con 0x7f5dbc0ac400 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc392d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc0ae270 con 0x7f5dbc0ae630 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc392d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0af0f0 con 0x7f5dbc0b1360 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc1eca700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5dbc0ac400 0x7f5dbc0ac880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35624/0 (socket says 172.21.15.92:35624) 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc1eca700 1 -- 172.21.15.92:0/3100234969 learned_addr learned my addr 172.21.15.92:0/3100234969 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987306724 0 0) 0x7f5dbc0adde0 con 0x7f5dbc0ac400 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc16c9700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 0x7f5dbc0b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da80035b0 con 0x7f5dbc0ac400 2023-06-09T04:35:12.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5dc0ec8700 1 --2- 172.21.15.92:0/3100234969 >> v2:172.21.15.123:3300/0 conn(0x7f5dbc0ae630 0x7f5dbc0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1198377557 0 0) 0x7f5da80035b0 con 0x7f5dbc0ac400 2023-06-09T04:35:12.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 msgr2=0x7f5dbc0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:12.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 0x7f5dbc0b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 >> v2:172.21.15.123:3300/0 conn(0x7f5dbc0ae630 msgr2=0x7f5dbc0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:12.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 --2- 172.21.15.92:0/3100234969 >> v2:172.21.15.123:3300/0 conn(0x7f5dbc0ae630 0x7f5dbc0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0b1c70 con 0x7f5dbc0ac400 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.157+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f5db40019f0 con 0x7f5dbc0ac400 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 >> v1:172.21.15.92:6789/0 conn(0x7f5dbc0ac400 legacy=0x7f5dbc0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 0x7f5da8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8003f80 0x7f5da8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da8004d90 0x7f5da8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0ae270 con 0x7f5da8004d90 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0af0f0 con 0x7f5da8003f80 2023-06-09T04:35:12.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da800ba10 con 0x7f5dbc0b1360 2023-06-09T04:35:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc1eca700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da8004d90 0x7f5da8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc0ec8700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8003f80 0x7f5da8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc16c9700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 0x7f5da8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc1eca700 1 -- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 msgr2=0x7f5da8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc1eca700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc0b1360 0x7f5da8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc1eca700 1 -- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8003f80 msgr2=0x7f5da8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:12.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc1eca700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8003f80 0x7f5da8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc1eca700 1 -- 172.21.15.92:0/3100234969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da80035b0 con 0x7f5da8004d90 2023-06-09T04:35:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5dc1eca700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da8004d90 0x7f5da8005200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5db400a8d0 tx=0x7f5da8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d6b27b856624be server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400a340 con 0x7f5da8004d90 2023-06-09T04:35:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4012600 con 0x7f5da8004d90 2023-06-09T04:35:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.158+0000 7f5db3fff700 1 -- 172.21.15.92:0/3100234969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40129b0 con 0x7f5da8004d90 2023-06-09T04:35:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.159+0000 7f5dc392d700 1 -- 172.21.15.92:0/3100234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da8004d90 msgr2=0x7f5da8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.159+0000 7f5dc392d700 1 --2- 172.21.15.92:0/3100234969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da8004d90 0x7f5da8005200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5db400a8d0 tx=0x7f5da8005880 comp rx=0 tx=0).stop 2023-06-09T04:35:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.159+0000 7f5dc1eca700 1 -- 172.21.15.92:0/3100234969 reap_dead start 2023-06-09T04:35:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.159+0000 7f5dc392d700 1 -- 172.21.15.92:0/3100234969 shutdown_connections 2023-06-09T04:35:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.159+0000 7f5dc392d700 1 -- 172.21.15.92:0/3100234969 >> 172.21.15.92:0/3100234969 conn(0x7f5dbc098c10 msgr2=0x7f5dbc0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.159+0000 7f5dc392d700 1 -- 172.21.15.92:0/3100234969 shutdown_connections 2023-06-09T04:35:12.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.159+0000 7f5dc392d700 1 -- 172.21.15.92:0/3100234969 wait complete. 2023-06-09T04:35:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.160+0000 7f5dc392d700 1 Processor -- start 2023-06-09T04:35:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.160+0000 7f5dc392d700 1 -- start start 2023-06-09T04:35:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc392d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8004d90 0x7f5dbc0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc392d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc14f030 0x7f5dbc14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc392d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dbc0c7c90 0x7f5dbc0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc392d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0c9700 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc392d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc14f9e0 con 0x7f5da8004d90 2023-06-09T04:35:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc392d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0c8a80 con 0x7f5dbc14f030 2023-06-09T04:35:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dbc0c7c90 0x7f5dbc0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dbc0c7c90 0x7f5dbc0c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37302/0 (socket says 172.21.15.92:37302) 2023-06-09T04:35:12.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 -- 172.21.15.92:0/2882602950 learned_addr learned my addr 172.21.15.92:0/2882602950 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc0ec8700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc14f030 0x7f5dbc14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc16c9700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8004d90 0x7f5dbc0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 -- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc14f030 msgr2=0x7f5dbc14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc14f030 0x7f5dbc14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 -- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8004d90 msgr2=0x7f5dbc0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8004d90 0x7f5dbc0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 -- 172.21.15.92:0/2882602950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0c8d60 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc16c9700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8004d90 0x7f5dbc0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:12.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5dc1eca700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dbc0c7c90 0x7f5dbc0c8130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5db4015040 tx=0x7f5db4001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.161+0000 7f5db1ffb700 1 -- 172.21.15.92:0/2882602950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400a340 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.162+0000 7f5db1ffb700 1 -- 172.21.15.92:0/2882602950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db40019c0 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.162+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc0c9040 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.162+0000 7f5db1ffb700 1 -- 172.21.15.92:0/2882602950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4026640 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.162+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc0c8670 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.163+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc165030 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.163+0000 7f5db1ffb700 1 -- 172.21.15.92:0/2882602950 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5db4019030 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.163+0000 7f5db1ffb700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5da006ace0 0x7f5da006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.164+0000 7f5dc16c9700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5da006ace0 0x7f5da006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.164+0000 7f5db1ffb700 1 -- 172.21.15.92:0/2882602950 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5db408a990 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.164+0000 7f5dc16c9700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5da006ace0 0x7f5da006d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009c40 tx=0x7f5dac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.168+0000 7f5db1ffb700 1 -- 172.21.15.92:0/2882602950 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5dbc165030 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.357+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dbc0c9690 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.357+0000 7f5db1ffb700 1 -- 172.21.15.92:0/2882602950 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14625 (secure 0 0 0) 0x7f5db4017020 con 0x7f5dbc0c7c90 2023-06-09T04:35:12.359 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:12.359 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:11.986214+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:12.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5da006ace0 msgr2=0x7f5da006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5da006ace0 0x7f5da006d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009c40 tx=0x7f5dac008040 comp rx=0 tx=0).stop 2023-06-09T04:35:12.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dbc0c7c90 msgr2=0x7f5dbc0c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dbc0c7c90 0x7f5dbc0c8130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5db4015040 tx=0x7f5db4001360 comp rx=0 tx=0).stop 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 shutdown_connections 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5da006ace0 0x7f5da006d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5dbc0c7c90 0x7f5dbc0c8130 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5da8004d90 0x7f5dbc0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 --2- 172.21.15.92:0/2882602950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5dbc14f030 0x7f5dbc14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 >> 172.21.15.92:0/2882602950 conn(0x7f5dbc098c10 msgr2=0x7f5dbc0c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:12.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 shutdown_connections 2023-06-09T04:35:12.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.360+0000 7f5dc392d700 1 -- 172.21.15.92:0/2882602950 wait complete. 2023-06-09T04:35:12.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:12.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.493+0000 7f6a883df700 1 Processor -- start 2023-06-09T04:35:12.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.494+0000 7f6a883df700 1 -- start start 2023-06-09T04:35:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.494+0000 7f6a883df700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a800ce3d0 0x7f6a800713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a883df700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a800aed50 0x7f6a800ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a883df700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800ce190 con 0x7f6a8015d470 2023-06-09T04:35:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a883df700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a80072350 con 0x7f6a800ce3d0 2023-06-09T04:35:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a883df700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800cece0 con 0x7f6a800aed50 2023-06-09T04:35:12.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a800aed50 0x7f6a800ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a800aed50 0x7f6a800ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46220/0 (socket says 172.21.15.92:46220) 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8617b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6a800ce3d0 0x7f6a800713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8697c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6a8015d470 0x7f6a801596e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58140/0 (socket says 172.21.15.92:58140) 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 -- 172.21.15.92:0/3022348442 learned_addr learned my addr 172.21.15.92:0/3022348442 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a85179700 1 -- 172.21.15.92:0/3022348442 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981346858 0 0) 0x7f6a800ce190 con 0x7f6a8015d470 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a85179700 1 -- 172.21.15.92:0/3022348442 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a6c0035b0 con 0x7f6a8015d470 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 -- 172.21.15.92:0/3022348442 >> v2:172.21.15.123:3300/0 conn(0x7f6a800ce3d0 msgr2=0x7f6a800713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:12.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 --2- 172.21.15.92:0/3022348442 >> v2:172.21.15.123:3300/0 conn(0x7f6a800ce3d0 0x7f6a800713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 -- 172.21.15.92:0/3022348442 >> v1:172.21.15.123:6789/0 conn(0x7f6a8015d470 legacy=0x7f6a801596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 -- 172.21.15.92:0/3022348442 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80071ab0 con 0x7f6a800aed50 2023-06-09T04:35:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8617b700 1 --2- 172.21.15.92:0/3022348442 >> v2:172.21.15.123:3300/0 conn(0x7f6a800ce3d0 0x7f6a800713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a8597a700 1 --2- 172.21.15.92:0/3022348442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a800aed50 0x7f6a800ce7a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098f0 tx=0x7f6a70009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bdd38c3a1ffc76d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a85179700 1 -- 172.21.15.92:0/3022348442 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011830 con 0x7f6a800aed50 2023-06-09T04:35:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.495+0000 7f6a85179700 1 -- 172.21.15.92:0/3022348442 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a70011990 con 0x7f6a800aed50 2023-06-09T04:35:12.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a85179700 1 -- 172.21.15.92:0/3022348442 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011d00 con 0x7f6a800aed50 2023-06-09T04:35:12.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 -- 172.21.15.92:0/3022348442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a800aed50 msgr2=0x7f6a800ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 --2- 172.21.15.92:0/3022348442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a800aed50 0x7f6a800ce7a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098f0 tx=0x7f6a70009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 -- 172.21.15.92:0/3022348442 shutdown_connections 2023-06-09T04:35:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 --2- 172.21.15.92:0/3022348442 >> v2:172.21.15.123:3300/0 conn(0x7f6a800ce3d0 0x7f6a800713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 --2- 172.21.15.92:0/3022348442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a800aed50 0x7f6a800ce7a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 -- 172.21.15.92:0/3022348442 >> 172.21.15.92:0/3022348442 conn(0x7f6a800cc380 msgr2=0x7f6a800cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 -- 172.21.15.92:0/3022348442 shutdown_connections 2023-06-09T04:35:12.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.496+0000 7f6a883df700 1 -- 172.21.15.92:0/3022348442 wait complete. 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.497+0000 7f6a883df700 1 Processor -- start 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.497+0000 7f6a883df700 1 -- start start 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a883df700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a800aed50 0x7f6a80151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a883df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a800ce3d0 0x7f6a80069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a883df700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8015d470 0x7f6a800c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a883df700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800c7f30 con 0x7f6a800ce3d0 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a883df700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8006acc0 con 0x7f6a800aed50 2023-06-09T04:35:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a883df700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8006a700 con 0x7f6a8015d470 2023-06-09T04:35:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a800ce3d0 0x7f6a80069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a800ce3d0 0x7f6a80069db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37316/0 (socket says 172.21.15.92:37316) 2023-06-09T04:35:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 -- 172.21.15.92:0/4276516448 learned_addr learned my addr 172.21.15.92:0/4276516448 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8697c700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8015d470 0x7f6a800c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8617b700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a800aed50 0x7f6a80151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 -- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8015d470 msgr2=0x7f6a800c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8015d470 0x7f6a800c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 -- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a800aed50 msgr2=0x7f6a80151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a800aed50 0x7f6a80151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.498+0000 7f6a8597a700 1 -- 172.21.15.92:0/4276516448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0094a0 con 0x7f6a800ce3d0 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.499+0000 7f6a8597a700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a800ce3d0 0x7f6a80069db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a70014040 tx=0x7f6a70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.499+0000 7f6a76ffd700 1 -- 172.21.15.92:0/4276516448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004590 con 0x7f6a800ce3d0 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.499+0000 7f6a76ffd700 1 -- 172.21.15.92:0/4276516448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7001e070 con 0x7f6a800ce3d0 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.499+0000 7f6a76ffd700 1 -- 172.21.15.92:0/4276516448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a700038e0 con 0x7f6a800ce3d0 2023-06-09T04:35:12.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.499+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a70009550 con 0x7f6a800ce3d0 2023-06-09T04:35:12.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.499+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a800c7b50 con 0x7f6a800ce3d0 2023-06-09T04:35:12.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.500+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8006be50 con 0x7f6a800ce3d0 2023-06-09T04:35:12.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.500+0000 7f6a76ffd700 1 -- 172.21.15.92:0/4276516448 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a70003a40 con 0x7f6a800ce3d0 2023-06-09T04:35:12.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.501+0000 7f6a76ffd700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a6406ace0 0x7f6a6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.501+0000 7f6a76ffd700 1 -- 172.21.15.92:0/4276516448 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6a7008a0a0 con 0x7f6a800ce3d0 2023-06-09T04:35:12.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.501+0000 7f6a8617b700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a6406ace0 0x7f6a6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.501+0000 7f6a8617b700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a6406ace0 0x7f6a6406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009d00 tx=0x7f6a7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.505+0000 7f6a76ffd700 1 -- 172.21.15.92:0/4276516448 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6a70055490 con 0x7f6a800ce3d0 2023-06-09T04:35:12.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.689+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a8004e8e0 con 0x7f6a800ce3d0 2023-06-09T04:35:12.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.689+0000 7f6a76ffd700 1 -- 172.21.15.92:0/4276516448 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a700200a0 con 0x7f6a800ce3d0 2023-06-09T04:35:12.691 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a6406ace0 msgr2=0x7f6a6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a6406ace0 0x7f6a6406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009d00 tx=0x7f6a7c008040 comp rx=0 tx=0).stop 2023-06-09T04:35:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a800ce3d0 msgr2=0x7f6a80069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a800ce3d0 0x7f6a80069db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a70014040 tx=0x7f6a70004750 comp rx=0 tx=0).stop 2023-06-09T04:35:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 shutdown_connections 2023-06-09T04:35:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a6406ace0 0x7f6a6406d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8015d470 0x7f6a800c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a800aed50 0x7f6a80151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 --2- 172.21.15.92:0/4276516448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a800ce3d0 0x7f6a80069db0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 >> 172.21.15.92:0/4276516448 conn(0x7f6a800cc380 msgr2=0x7f6a800b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:12.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 shutdown_connections 2023-06-09T04:35:12.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.691+0000 7f6a883df700 1 -- 172.21.15.92:0/4276516448 wait complete. 2023-06-09T04:35:12.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:12.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.825+0000 7f3c0a4ee700 1 Processor -- start 2023-06-09T04:35:12.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.825+0000 7f3c0a4ee700 1 -- start start 2023-06-09T04:35:12.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c0a4ee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c04069db0 0x7f3c0415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c0a4ee700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c040cf440 0x7f3c0406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c0a4ee700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0415d040 con 0x7f3c040cce60 2023-06-09T04:35:12.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c0a4ee700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c04160ab0 con 0x7f3c040cf440 2023-06-09T04:35:12.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c0a4ee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0415ca10 con 0x7f3c04069db0 2023-06-09T04:35:12.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c040cf440 0x7f3c0406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c040cf440 0x7f3c0406a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37324/0 (socket says 172.21.15.92:37324) 2023-06-09T04:35:12.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 -- 172.21.15.92:0/400522248 learned_addr learned my addr 172.21.15.92:0/400522248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:12.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c03fff700 1 --2- 172.21.15.92:0/400522248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c04069db0 0x7f3c0415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 -- 172.21.15.92:0/400522248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c04069db0 msgr2=0x7f3c0415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:12.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 --2- 172.21.15.92:0/400522248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c04069db0 0x7f3c0415d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 -- 172.21.15.92:0/400522248 >> v1:172.21.15.123:6789/0 conn(0x7f3c040cce60 legacy=0x7f3c0406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 -- 172.21.15.92:0/400522248 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0406a6c0 con 0x7f3c040cf440 2023-06-09T04:35:12.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c037fe700 1 --2- 172.21.15.92:0/400522248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c0406a180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40098f0 tx=0x7f3bf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83692b7fcdab23b1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c02ffd700 1 -- 172.21.15.92:0/400522248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4011830 con 0x7f3c040cf440 2023-06-09T04:35:12.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c02ffd700 1 -- 172.21.15.92:0/400522248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf4011990 con 0x7f3c040cf440 2023-06-09T04:35:12.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.826+0000 7f3c02ffd700 1 -- 172.21.15.92:0/400522248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4011d20 con 0x7f3c040cf440 2023-06-09T04:35:12.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/400522248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 msgr2=0x7f3c0406a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/400522248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c0406a180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40098f0 tx=0x7f3bf4009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:12.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/400522248 shutdown_connections 2023-06-09T04:35:12.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/400522248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c04069db0 0x7f3c0415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/400522248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c0406a180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/400522248 >> 172.21.15.92:0/400522248 conn(0x7f3c040cc380 msgr2=0x7f3c040cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:12.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/400522248 shutdown_connections 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.827+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/400522248 wait complete. 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.828+0000 7f3c0a4ee700 1 Processor -- start 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.828+0000 7f3c0a4ee700 1 -- start start 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c0a4ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c04069db0 0x7f3c040c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c0a4ee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c040cce60 0x7f3c04155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c0a4ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c040af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c0a4ee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040c49b0 con 0x7f3c040cf440 2023-06-09T04:35:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c0a4ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040c7fd0 con 0x7f3c04069db0 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c0a4ee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c040c46a0 con 0x7f3c040cce60 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c08a8b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c040af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c08a8b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c040af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37332/0 (socket says 172.21.15.92:37332) 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c08a8b700 1 -- 172.21.15.92:0/3153133903 learned_addr learned my addr 172.21.15.92:0/3153133903 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c037fe700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c040cce60 0x7f3c04155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.829+0000 7f3c03fff700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c04069db0 0x7f3c040c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c08a8b700 1 -- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c040cce60 msgr2=0x7f3c04155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c08a8b700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c040cce60 0x7f3c04155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c08a8b700 1 -- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c04069db0 msgr2=0x7f3c040c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:12.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c08a8b700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c04069db0 0x7f3c040c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:12.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c08a8b700 1 -- 172.21.15.92:0/3153133903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf4009550 con 0x7f3c040cf440 2023-06-09T04:35:12.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c08a8b700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c040af340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00cb90 tx=0x7f3bfc00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c00ff9700 1 -- 172.21.15.92:0/3153133903 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc003c00 con 0x7f3c040cf440 2023-06-09T04:35:12.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c00ff9700 1 -- 172.21.15.92:0/3153133903 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc003d60 con 0x7f3c040cf440 2023-06-09T04:35:12.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c00ff9700 1 -- 172.21.15.92:0/3153133903 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00aae0 con 0x7f3c040cf440 2023-06-09T04:35:12.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c04155b80 con 0x7f3c040cf440 2023-06-09T04:35:12.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.830+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c040c6a70 con 0x7f3c040cf440 2023-06-09T04:35:12.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.831+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c0415ccb0 con 0x7f3c040cf440 2023-06-09T04:35:12.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.831+0000 7f3c00ff9700 1 -- 172.21.15.92:0/3153133903 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3bfc00ac40 con 0x7f3c040cf440 2023-06-09T04:35:12.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.832+0000 7f3c00ff9700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3bf006ace0 0x7f3bf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:12.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.832+0000 7f3c03fff700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3bf006ace0 0x7f3bf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:12.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.832+0000 7f3c00ff9700 1 -- 172.21.15.92:0/3153133903 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3bfc089070 con 0x7f3c040cf440 2023-06-09T04:35:12.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.832+0000 7f3c03fff700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3bf006ace0 0x7f3bf006d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3c040c6410 tx=0x7f3bf8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:12.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:12.836+0000 7f3c00ff9700 1 -- 172.21.15.92:0/3153133903 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3c0415ccb0 con 0x7f3c040cf440 2023-06-09T04:35:13.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.026+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3c0415ccb0 con 0x7f3c040cf440 2023-06-09T04:35:13.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.026+0000 7f3c00ff9700 1 -- 172.21.15.92:0/3153133903 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3c0415ccb0 con 0x7f3c040cf440 2023-06-09T04:35:13.028 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3bf006ace0 msgr2=0x7f3bf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3bf006ace0 0x7f3bf006d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3c040c6410 tx=0x7f3bf8009210 comp rx=0 tx=0).stop 2023-06-09T04:35:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 msgr2=0x7f3c040af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c040af340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00cb90 tx=0x7f3bfc00cea0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 shutdown_connections 2023-06-09T04:35:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3bf006ace0 0x7f3bf006d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c040cf440 0x7f3c040af340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c04069db0 0x7f3c040c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 --2- 172.21.15.92:0/3153133903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c040cce60 0x7f3c04155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.028+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 >> 172.21.15.92:0/3153133903 conn(0x7f3c040cc380 msgr2=0x7f3c0415b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.029+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 shutdown_connections 2023-06-09T04:35:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.029+0000 7f3c0a4ee700 1 -- 172.21.15.92:0/3153133903 wait complete. 2023-06-09T04:35:13.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:35:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.161+0000 7f8490531700 1 Processor -- start 2023-06-09T04:35:13.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.162+0000 7f8490531700 1 -- start start 2023-06-09T04:35:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.162+0000 7f8490531700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 0x7f84880ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f8490531700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84880ce730 con 0x7f848815c7d0 2023-06-09T04:35:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f8490531700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848806aaf0 con 0x7f84880cf010 2023-06-09T04:35:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f8490531700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848815cba0 con 0x7f84880ac400 2023-06-09T04:35:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848e2cd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f848815c7d0 0x7f84880ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35672/0 (socket says 172.21.15.92:35672) 2023-06-09T04:35:13.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848e2cd700 1 -- 172.21.15.92:0/3068722505 learned_addr learned my addr 172.21.15.92:0/3068722505 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848eace700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 0x7f84880ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451547576 0 0) 0x7f8478001290 con 0x7f848815c7d0 2023-06-09T04:35:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84740035b0 con 0x7f848815c7d0 2023-06-09T04:35:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1262219758 0 0) 0x7f84740035b0 con 0x7f848815c7d0 2023-06-09T04:35:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 msgr2=0x7f84880ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 0x7f84880ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 >> v1:172.21.15.123:6789/0 conn(0x7f84880cf010 legacy=0x7f848806e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488069ee0 con 0x7f848815c7d0 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f8478001980 con 0x7f848815c7d0 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848eace700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 0x7f84880ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 >> v1:172.21.15.92:6789/0 conn(0x7f848815c7d0 legacy=0x7f84880ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8474003a30 0x7f8474003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84740043d0 0x7f8474004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 0x7f8474005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848815cba0 con 0x7f8474003a30 2023-06-09T04:35:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847400ba70 con 0x7f84740043d0 2023-06-09T04:35:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847400bc30 con 0x7f84880ac400 2023-06-09T04:35:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848e2cd700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8474003a30 0x7f8474003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848eace700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 0x7f8474005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.163+0000 7f848dacc700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84740043d0 0x7f8474004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848e2cd700 1 -- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 msgr2=0x7f8474005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848e2cd700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880ac400 0x7f8474005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848e2cd700 1 -- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84740043d0 msgr2=0x7f8474004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848e2cd700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84740043d0 0x7f8474004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848e2cd700 1 -- 172.21.15.92:0/3068722505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480009a70 con 0x7f8474003a30 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848dacc700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84740043d0 0x7f8474004890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848e2cd700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8474003a30 0x7f8474003e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f847800c240 tx=0x7f8474005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266c7d1a9f6163df server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800cde0 con 0x7f8474003a30 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478005830 con 0x7f8474003a30 2023-06-09T04:35:13.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848d2cb700 1 -- 172.21.15.92:0/3068722505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478005be0 con 0x7f8474003a30 2023-06-09T04:35:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f8490531700 1 -- 172.21.15.92:0/3068722505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8474003a30 msgr2=0x7f8474003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f8490531700 1 --2- 172.21.15.92:0/3068722505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8474003a30 0x7f8474003e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f847800c240 tx=0x7f8474005880 comp rx=0 tx=0).stop 2023-06-09T04:35:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f848eace700 1 -- 172.21.15.92:0/3068722505 reap_dead start 2023-06-09T04:35:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f8490531700 1 -- 172.21.15.92:0/3068722505 shutdown_connections 2023-06-09T04:35:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f8490531700 1 -- 172.21.15.92:0/3068722505 >> 172.21.15.92:0/3068722505 conn(0x7f8488098c10 msgr2=0x7f84880cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f8490531700 1 -- 172.21.15.92:0/3068722505 shutdown_connections 2023-06-09T04:35:13.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.164+0000 7f8490531700 1 -- 172.21.15.92:0/3068722505 wait complete. 2023-06-09T04:35:13.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 Processor -- start 2023-06-09T04:35:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 -- start start 2023-06-09T04:35:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8474003a30 0x7f84880c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f848815b0f0 0x7f848814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880c50a0 0x7f84880c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c5a50 con 0x7f848815b0f0 2023-06-09T04:35:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880cb720 con 0x7f8474003a30 2023-06-09T04:35:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f8490531700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c6aa0 con 0x7f84880c50a0 2023-06-09T04:35:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f848dacc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f848815b0f0 0x7f848814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f848dacc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f848815b0f0 0x7f848814ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37354/0 (socket says 172.21.15.92:37354) 2023-06-09T04:35:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f848dacc700 1 -- 172.21.15.92:0/108255942 learned_addr learned my addr 172.21.15.92:0/108255942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.166+0000 7f848eace700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880c50a0 0x7f84880c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848dacc700 1 -- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880c50a0 msgr2=0x7f84880c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848dacc700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880c50a0 0x7f84880c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848dacc700 1 -- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8474003a30 msgr2=0x7f84880c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848e2cd700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8474003a30 0x7f84880c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848dacc700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8474003a30 0x7f84880c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848dacc700 1 -- 172.21.15.92:0/108255942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880c6d80 con 0x7f848815b0f0 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848e2cd700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8474003a30 0x7f84880c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f848dacc700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f848815b0f0 0x7f848814ac00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84840079d0 tx=0x7f848400ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f847effd700 1 -- 172.21.15.92:0/108255942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84840052e0 con 0x7f848815b0f0 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f847effd700 1 -- 172.21.15.92:0/108255942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484005440 con 0x7f848815b0f0 2023-06-09T04:35:13.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84880cb970 con 0x7f848815b0f0 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f847effd700 1 -- 172.21.15.92:0/108255942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84840038a0 con 0x7f848815b0f0 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.167+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84880c9be0 con 0x7f848815b0f0 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.168+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84880740e0 con 0x7f848815b0f0 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.168+0000 7f847effd700 1 -- 172.21.15.92:0/108255942 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8484003a00 con 0x7f848815b0f0 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.169+0000 7f847effd700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f846c06ace0 0x7f846c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.169+0000 7f847effd700 1 -- 172.21.15.92:0/108255942 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f84840886f0 con 0x7f848815b0f0 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.169+0000 7f848e2cd700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f846c06ace0 0x7f846c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.169+0000 7f848e2cd700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f846c06ace0 0x7f846c06d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8478000ef0 tx=0x7f8478007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.173+0000 7f847effd700 1 -- 172.21.15.92:0/108255942 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8484053ba0 con 0x7f848815b0f0 2023-06-09T04:35:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.399+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84880ced10 con 0x7f848815b0f0 2023-06-09T04:35:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.400+0000 7f847effd700 1 -- 172.21.15.92:0/108255942 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f8484019de0 con 0x7f848815b0f0 2023-06-09T04:35:13.401 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:13.402 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-09T04:35:13.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f846c06ace0 msgr2=0x7f846c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f846c06ace0 0x7f846c06d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8478000ef0 tx=0x7f8478007000 comp rx=0 tx=0).stop 2023-06-09T04:35:13.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f848815b0f0 msgr2=0x7f848814ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f848815b0f0 0x7f848814ac00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84840079d0 tx=0x7f848400ea60 comp rx=0 tx=0).stop 2023-06-09T04:35:13.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 shutdown_connections 2023-06-09T04:35:13.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f846c06ace0 0x7f846c06d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84880c50a0 0x7f84880c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8474003a30 0x7f84880c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 --2- 172.21.15.92:0/108255942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f848815b0f0 0x7f848814ac00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 >> 172.21.15.92:0/108255942 conn(0x7f8488098c10 msgr2=0x7f84880cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 shutdown_connections 2023-06-09T04:35:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.402+0000 7f8490531700 1 -- 172.21.15.92:0/108255942 wait complete. 2023-06-09T04:35:13.405 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2023-06-09T04:35:13.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:13.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.533+0000 7f100eafa700 1 Processor -- start 2023-06-09T04:35:13.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.533+0000 7f100eafa700 1 -- start start 2023-06-09T04:35:13.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100eafa700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10080ac400 0x7f10080ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100eafa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10080ce730 con 0x7f100815c7d0 2023-06-09T04:35:13.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100eafa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100806aaf0 con 0x7f10080cf010 2023-06-09T04:35:13.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100eafa700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100815cba0 con 0x7f10080ac400 2023-06-09T04:35:13.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100c896700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f100815c7d0 0x7f10080ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35692/0 (socket says 172.21.15.92:35692) 2023-06-09T04:35:13.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100c896700 1 -- 172.21.15.92:0/3605991247 learned_addr learned my addr 172.21.15.92:0/3605991247 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:13.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100d097700 1 --2- 172.21.15.92:0/3605991247 >> v2:172.21.15.123:3300/0 conn(0x7f10080ac400 0x7f10080ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054217985 0 0) 0x7f0ffc001290 con 0x7f100815c7d0 2023-06-09T04:35:13.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff80035b0 con 0x7f100815c7d0 2023-06-09T04:35:13.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006263389 0 0) 0x7f0ff80035b0 con 0x7f100815c7d0 2023-06-09T04:35:13.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 >> v2:172.21.15.123:3300/0 conn(0x7f10080ac400 msgr2=0x7f10080ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 --2- 172.21.15.92:0/3605991247 >> v2:172.21.15.123:3300/0 conn(0x7f10080ac400 0x7f10080ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 >> v1:172.21.15.123:6789/0 conn(0x7f10080cf010 legacy=0x7f100806e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008069ee0 con 0x7f100815c7d0 2023-06-09T04:35:13.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f0ffc001980 con 0x7f100815c7d0 2023-06-09T04:35:13.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 >> v1:172.21.15.92:6789/0 conn(0x7f100815c7d0 legacy=0x7f10080ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ff8003a00 0x7f0ff8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8004850 0x7f0ff8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ff8005320 0x7f0ff800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100815cba0 con 0x7f0ff8005320 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff800c1e0 con 0x7f0ff8004850 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff800c3a0 con 0x7f0ff8003a00 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.534+0000 7f100d097700 1 --2- 172.21.15.92:0/3605991247 >> v2:172.21.15.123:3300/0 conn(0x7f10080ac400 0x7f10080ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ff8005320 0x7f0ff800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100c896700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ff8003a00 0x7f0ff8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f1007fff700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8004850 0x7f0ff8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 -- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ff8003a00 msgr2=0x7f0ff8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ff8003a00 0x7f0ff8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 -- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8004850 msgr2=0x7f0ff8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8004850 0x7f0ff8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 -- 172.21.15.92:0/3605991247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80035b0 con 0x7f0ff8005320 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 -- 172.21.15.92:0/3605991247 reap_dead start 2023-06-09T04:35:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f100d097700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ff8005320 0x7f0ff800ba10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10080cd3d0 tx=0x7f0ff8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fe99c2244816e2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000009b20 con 0x7f0ff8005320 2023-06-09T04:35:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000003a20 con 0x7f0ff8005320 2023-06-09T04:35:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.535+0000 7f10077fe700 1 -- 172.21.15.92:0/3605991247 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000003dd0 con 0x7f0ff8005320 2023-06-09T04:35:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.536+0000 7f100eafa700 1 -- 172.21.15.92:0/3605991247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ff8005320 msgr2=0x7f0ff800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.536+0000 7f100eafa700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ff8005320 0x7f0ff800ba10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10080cd3d0 tx=0x7f0ff8005880 comp rx=0 tx=0).stop 2023-06-09T04:35:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.536+0000 7f100eafa700 1 -- 172.21.15.92:0/3605991247 shutdown_connections 2023-06-09T04:35:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.536+0000 7f100eafa700 1 --2- 172.21.15.92:0/3605991247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ff8005320 0x7f0ff800ba10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.536+0000 7f100eafa700 1 -- 172.21.15.92:0/3605991247 >> 172.21.15.92:0/3605991247 conn(0x7f1008098c10 msgr2=0x7f10080cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:13.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.536+0000 7f100eafa700 1 -- 172.21.15.92:0/3605991247 shutdown_connections 2023-06-09T04:35:13.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.536+0000 7f100eafa700 1 -- 172.21.15.92:0/3605991247 wait complete. 2023-06-09T04:35:13.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.537+0000 7f100eafa700 1 Processor -- start 2023-06-09T04:35:13.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.537+0000 7f100eafa700 1 -- start start 2023-06-09T04:35:13.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100eafa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8005320 0x7f10080b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100eafa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10081f5340 0x7f10081f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100eafa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10080c4db0 0x7f10080c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100eafa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1008099580 con 0x7f10080c4db0 2023-06-09T04:35:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100eafa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1008099860 con 0x7f0ff8005320 2023-06-09T04:35:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100eafa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10080c5760 con 0x7f10081f5340 2023-06-09T04:35:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10080c4db0 0x7f10080c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10080c4db0 0x7f10080c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37376/0 (socket says 172.21.15.92:37376) 2023-06-09T04:35:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 -- 172.21.15.92:0/4157986906 learned_addr learned my addr 172.21.15.92:0/4157986906 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f1007fff700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10081f5340 0x7f10081f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 -- 172.21.15.92:0/4157986906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10081f5340 msgr2=0x7f10081f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10081f5340 0x7f10081f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 -- 172.21.15.92:0/4157986906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8005320 msgr2=0x7f10080b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8005320 0x7f10080b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 -- 172.21.15.92:0/4157986906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10081f1b90 con 0x7f10080c4db0 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100d097700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10080c4db0 0x7f10080c5220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1000007970 tx=0x7f1000004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f10057fa700 1 -- 172.21.15.92:0/4157986906 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000007000 con 0x7f10080c4db0 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f10057fa700 1 -- 172.21.15.92:0/4157986906 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000009ac0 con 0x7f10080c4db0 2023-06-09T04:35:13.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.538+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10081f5b50 con 0x7f10080c4db0 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.539+0000 7f10057fa700 1 -- 172.21.15.92:0/4157986906 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100001f660 con 0x7f10080c4db0 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.539+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10081f6110 con 0x7f10080c4db0 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.539+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10081f62a0 con 0x7f10080c4db0 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.540+0000 7f10057fa700 1 -- 172.21.15.92:0/4157986906 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f100001f7c0 con 0x7f10080c4db0 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.541+0000 7f10057fa700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0fec06ace0 0x7f0fec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.541+0000 7f100c896700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0fec06ace0 0x7f0fec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.541+0000 7f10057fa700 1 -- 172.21.15.92:0/4157986906 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f100008b5e0 con 0x7f10080c4db0 2023-06-09T04:35:13.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.541+0000 7f100c896700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0fec06ace0 0x7f0fec06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0011d0 tx=0x7f0ffc007f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.545+0000 7f10057fa700 1 -- 172.21.15.92:0/4157986906 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1000056b50 con 0x7f10080c4db0 2023-06-09T04:35:13.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.733+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10080999a0 con 0x7f10080c4db0 2023-06-09T04:35:13.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.734+0000 7f10057fa700 1 -- 172.21.15.92:0/4157986906 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14612 (secure 0 0 0) 0x7f10080999a0 con 0x7f10080c4db0 2023-06-09T04:35:13.735 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:13.735 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":20,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:12.992299+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:13.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0fec06ace0 msgr2=0x7f0fec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0fec06ace0 0x7f0fec06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0011d0 tx=0x7f0ffc007f00 comp rx=0 tx=0).stop 2023-06-09T04:35:13.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10080c4db0 msgr2=0x7f10080c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10080c4db0 0x7f10080c5220 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1000007970 tx=0x7f1000004b40 comp rx=0 tx=0).stop 2023-06-09T04:35:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 shutdown_connections 2023-06-09T04:35:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0fec06ace0 0x7f0fec06d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10080c4db0 0x7f10080c5220 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ff8005320 0x7f10080b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 --2- 172.21.15.92:0/4157986906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10081f5340 0x7f10081f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 >> 172.21.15.92:0/4157986906 conn(0x7f1008098c10 msgr2=0x7f10080ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:13.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 shutdown_connections 2023-06-09T04:35:13.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.736+0000 7f100eafa700 1 -- 172.21.15.92:0/4157986906 wait complete. 2023-06-09T04:35:13.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:13.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e65e92700 1 Processor -- start 2023-06-09T04:35:13.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e65e92700 1 -- start start 2023-06-09T04:35:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e65e92700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8e600aea90 0x7f8e600cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e65e92700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e600aee60 0x7f8e60069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e65e92700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e60071dc0 con 0x7f8e6015e840 2023-06-09T04:35:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e65e92700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6015edf0 con 0x7f8e600aee60 2023-06-09T04:35:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e65e92700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6006fe30 con 0x7f8e600aea90 2023-06-09T04:35:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e5effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e600aee60 0x7f8e60069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.871+0000 7f8e5effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e600aee60 0x7f8e60069db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37380/0 (socket says 172.21.15.92:37380) 2023-06-09T04:35:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5effd700 1 -- 172.21.15.92:0/1659347070 learned_addr learned my addr 172.21.15.92:0/1659347070 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:13.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5f7fe700 1 --2- 172.21.15.92:0/1659347070 >> v2:172.21.15.123:3300/0 conn(0x7f8e600aea90 0x7f8e600cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5effd700 1 -- 172.21.15.92:0/1659347070 >> v2:172.21.15.123:3300/0 conn(0x7f8e600aea90 msgr2=0x7f8e600cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5effd700 1 --2- 172.21.15.92:0/1659347070 >> v2:172.21.15.123:3300/0 conn(0x7f8e600aea90 0x7f8e600cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5effd700 1 -- 172.21.15.92:0/1659347070 >> v1:172.21.15.123:6789/0 conn(0x7f8e6015e840 legacy=0x7f8e600b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5effd700 1 -- 172.21.15.92:0/1659347070 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600cf030 con 0x7f8e600aee60 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5effd700 1 --2- 172.21.15.92:0/1659347070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e600aee60 0x7f8e60069db0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e600710a0 tx=0x7f8e50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a78a7cf04c1244b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5e7fc700 1 -- 172.21.15.92:0/1659347070 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50004230 con 0x7f8e600aee60 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5e7fc700 1 -- 172.21.15.92:0/1659347070 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e50004390 con 0x7f8e600aee60 2023-06-09T04:35:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e5e7fc700 1 -- 172.21.15.92:0/1659347070 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e500036a0 con 0x7f8e600aee60 2023-06-09T04:35:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.872+0000 7f8e65e92700 1 -- 172.21.15.92:0/1659347070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e600aee60 msgr2=0x7f8e60069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.873+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1659347070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e600aee60 0x7f8e60069db0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e600710a0 tx=0x7f8e50009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.873+0000 7f8e65e92700 1 -- 172.21.15.92:0/1659347070 shutdown_connections 2023-06-09T04:35:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.873+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1659347070 >> v2:172.21.15.123:3300/0 conn(0x7f8e600aea90 0x7f8e600cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.873+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1659347070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e600aee60 0x7f8e60069db0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.873+0000 7f8e65e92700 1 -- 172.21.15.92:0/1659347070 >> 172.21.15.92:0/1659347070 conn(0x7f8e600cc380 msgr2=0x7f8e600cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:13.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.873+0000 7f8e65e92700 1 -- 172.21.15.92:0/1659347070 shutdown_connections 2023-06-09T04:35:13.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.873+0000 7f8e65e92700 1 -- 172.21.15.92:0/1659347070 wait complete. 2023-06-09T04:35:13.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.874+0000 7f8e65e92700 1 Processor -- start 2023-06-09T04:35:13.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.874+0000 7f8e65e92700 1 -- start start 2023-06-09T04:35:13.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e600aea90 0x7f8e600c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e6015e840 0x7f8e6014cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e600c7a20 0x7f8e601e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e600c8b30 con 0x7f8e6015e840 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e600c7eb0 con 0x7f8e600c7a20 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e600c8190 con 0x7f8e600aea90 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e6015e840 0x7f8e6014cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e6015e840 0x7f8e6014cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37386/0 (socket says 172.21.15.92:37386) 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 -- 172.21.15.92:0/1297801434 learned_addr learned my addr 172.21.15.92:0/1297801434 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5f7fe700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e600aea90 0x7f8e600c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5ffff700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e600c7a20 0x7f8e601e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 -- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e600aea90 msgr2=0x7f8e600c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e600aea90 0x7f8e600c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 -- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e600c7a20 msgr2=0x7f8e601e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e600c7a20 0x7f8e601e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 -- 172.21.15.92:0/1297801434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e50009550 con 0x7f8e6015e840 2023-06-09T04:35:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5ffff700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e600c7a20 0x7f8e601e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:13.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e5effd700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e6015e840 0x7f8e6014cc40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009130 tx=0x7f8e50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e57fff700 1 -- 172.21.15.92:0/1297801434 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50003d40 con 0x7f8e6015e840 2023-06-09T04:35:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e57fff700 1 -- 172.21.15.92:0/1297801434 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5001f070 con 0x7f8e6015e840 2023-06-09T04:35:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e600c8470 con 0x7f8e6015e840 2023-06-09T04:35:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e57fff700 1 -- 172.21.15.92:0/1297801434 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50011870 con 0x7f8e6015e840 2023-06-09T04:35:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.875+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6014d180 con 0x7f8e6015e840 2023-06-09T04:35:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.876+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e600c8c70 con 0x7f8e6015e840 2023-06-09T04:35:13.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.877+0000 7f8e57fff700 1 -- 172.21.15.92:0/1297801434 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e50021030 con 0x7f8e6015e840 2023-06-09T04:35:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.877+0000 7f8e57fff700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8e4006ace0 0x7f8e4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.877+0000 7f8e57fff700 1 -- 172.21.15.92:0/1297801434 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8e5008a5f0 con 0x7f8e6015e840 2023-06-09T04:35:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.877+0000 7f8e5f7fe700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8e4006ace0 0x7f8e4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:13.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.877+0000 7f8e5f7fe700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8e4006ace0 0x7f8e4006d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800b9b0 tx=0x7f8e48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:13.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:13.882+0000 7f8e57fff700 1 -- 172.21.15.92:0/1297801434 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8e600c8c70 con 0x7f8e6015e840 2023-06-09T04:35:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.069+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e600c8c70 con 0x7f8e6015e840 2023-06-09T04:35:14.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.070+0000 7f8e57fff700 1 -- 172.21.15.92:0/1297801434 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e600c8c70 con 0x7f8e6015e840 2023-06-09T04:35:14.071 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:14.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8e4006ace0 msgr2=0x7f8e4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8e4006ace0 0x7f8e4006d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800b9b0 tx=0x7f8e48008040 comp rx=0 tx=0).stop 2023-06-09T04:35:14.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e6015e840 msgr2=0x7f8e6014cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e6015e840 0x7f8e6014cc40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009130 tx=0x7f8e50004750 comp rx=0 tx=0).stop 2023-06-09T04:35:14.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 shutdown_connections 2023-06-09T04:35:14.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8e4006ace0 0x7f8e4006d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e600c7a20 0x7f8e601e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8e600aea90 0x7f8e600c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 --2- 172.21.15.92:0/1297801434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e6015e840 0x7f8e6014cc40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 >> 172.21.15.92:0/1297801434 conn(0x7f8e600cc380 msgr2=0x7f8e60069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 shutdown_connections 2023-06-09T04:35:14.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.072+0000 7f8e65e92700 1 -- 172.21.15.92:0/1297801434 wait complete. 2023-06-09T04:35:14.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:14.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.206+0000 7efd1e904700 1 Processor -- start 2023-06-09T04:35:14.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.206+0000 7efd1e904700 1 -- start start 2023-06-09T04:35:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd1e904700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efd180ce3d0 0x7efd180713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd1e904700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd180aed50 0x7efd180ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd1e904700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd180ce190 con 0x7efd1815d470 2023-06-09T04:35:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd1e904700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd18072350 con 0x7efd180ce3d0 2023-06-09T04:35:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd1e904700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd180cece0 con 0x7efd180aed50 2023-06-09T04:35:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efd180ce3d0 0x7efd180713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efd180ce3d0 0x7efd180713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37394/0 (socket says 172.21.15.92:37394) 2023-06-09T04:35:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 -- 172.21.15.92:0/3490933838 learned_addr learned my addr 172.21.15.92:0/3490933838 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 -- 172.21.15.92:0/3490933838 >> v2:172.21.15.123:3300/0 conn(0x7efd180aed50 msgr2=0x7efd180ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd177fe700 1 --2- 172.21.15.92:0/3490933838 >> v2:172.21.15.123:3300/0 conn(0x7efd180aed50 0x7efd180ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 --2- 172.21.15.92:0/3490933838 >> v2:172.21.15.123:3300/0 conn(0x7efd180aed50 0x7efd180ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 -- 172.21.15.92:0/3490933838 >> v1:172.21.15.123:6789/0 conn(0x7efd1815d470 legacy=0x7efd181596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:14.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 -- 172.21.15.92:0/3490933838 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18071ab0 con 0x7efd180ce3d0 2023-06-09T04:35:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd177fe700 1 --2- 172.21.15.92:0/3490933838 >> v2:172.21.15.123:3300/0 conn(0x7efd180aed50 0x7efd180ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.207+0000 7efd17fff700 1 --2- 172.21.15.92:0/3490933838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180ce3d0 0x7efd180713b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd180c33a0 tx=0x7efd0c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f75fad2d7779db server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd16ffd700 1 -- 172.21.15.92:0/3490933838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00dcf0 con 0x7efd180ce3d0 2023-06-09T04:35:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd16ffd700 1 -- 172.21.15.92:0/3490933838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c00de50 con 0x7efd180ce3d0 2023-06-09T04:35:14.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd16ffd700 1 -- 172.21.15.92:0/3490933838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c004250 con 0x7efd180ce3d0 2023-06-09T04:35:14.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 -- 172.21.15.92:0/3490933838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180ce3d0 msgr2=0x7efd180713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 --2- 172.21.15.92:0/3490933838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180ce3d0 0x7efd180713b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd180c33a0 tx=0x7efd0c00be70 comp rx=0 tx=0).stop 2023-06-09T04:35:14.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 -- 172.21.15.92:0/3490933838 shutdown_connections 2023-06-09T04:35:14.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 --2- 172.21.15.92:0/3490933838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180ce3d0 0x7efd180713b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 --2- 172.21.15.92:0/3490933838 >> v2:172.21.15.123:3300/0 conn(0x7efd180aed50 0x7efd180ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 -- 172.21.15.92:0/3490933838 >> 172.21.15.92:0/3490933838 conn(0x7efd180cc380 msgr2=0x7efd180cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 -- 172.21.15.92:0/3490933838 shutdown_connections 2023-06-09T04:35:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.208+0000 7efd1e904700 1 -- 172.21.15.92:0/3490933838 wait complete. 2023-06-09T04:35:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.210+0000 7efd1e904700 1 Processor -- start 2023-06-09T04:35:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.210+0000 7efd1e904700 1 -- start start 2023-06-09T04:35:14.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180aed50 0x7efd181f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd180ce3d0 0x7efd180c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1815d470 0x7efd18157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd180c7a10 con 0x7efd180aed50 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd180c9b40 con 0x7efd1815d470 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd180c6e00 con 0x7efd180ce3d0 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180aed50 0x7efd181f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180aed50 0x7efd181f1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37396/0 (socket says 172.21.15.92:37396) 2023-06-09T04:35:14.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 -- 172.21.15.92:0/813233485 learned_addr learned my addr 172.21.15.92:0/813233485 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd177fe700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd180ce3d0 0x7efd180c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 -- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd180ce3d0 msgr2=0x7efd180c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd180ce3d0 0x7efd180c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1cea1700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1815d470 0x7efd18157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 -- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1815d470 msgr2=0x7efd18157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1815d470 0x7efd18157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 -- 172.21.15.92:0/813233485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c00b7f0 con 0x7efd180aed50 2023-06-09T04:35:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1cea1700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1815d470 0x7efd18157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd17fff700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180aed50 0x7efd181f1e70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000c00 tx=0x7efd0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd14ff9700 1 -- 172.21.15.92:0/813233485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c020070 con 0x7efd180aed50 2023-06-09T04:35:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd14ff9700 1 -- 172.21.15.92:0/813233485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c004030 con 0x7efd180aed50 2023-06-09T04:35:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd14ff9700 1 -- 172.21.15.92:0/813233485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c010470 con 0x7efd180aed50 2023-06-09T04:35:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd180c70e0 con 0x7efd180aed50 2023-06-09T04:35:14.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.211+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd180c76a0 con 0x7efd180aed50 2023-06-09T04:35:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.212+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1815cf10 con 0x7efd180aed50 2023-06-09T04:35:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.213+0000 7efd14ff9700 1 -- 172.21.15.92:0/813233485 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd0c0193f0 con 0x7efd180aed50 2023-06-09T04:35:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.213+0000 7efd14ff9700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efd0406ace0 0x7efd0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.213+0000 7efd14ff9700 1 -- 172.21.15.92:0/813233485 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7efd0c08cb20 con 0x7efd180aed50 2023-06-09T04:35:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.213+0000 7efd177fe700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efd0406ace0 0x7efd0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.214+0000 7efd177fe700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efd0406ace0 0x7efd0406d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd08006fd0 tx=0x7efd08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.218+0000 7efd14ff9700 1 -- 172.21.15.92:0/813233485 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efd0c0580d0 con 0x7efd180aed50 2023-06-09T04:35:14.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.404+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efd1806be50 con 0x7efd180aed50 2023-06-09T04:35:14.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.404+0000 7efd14ff9700 1 -- 172.21.15.92:0/813233485 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7efd0c05b6f0 con 0x7efd180aed50 2023-06-09T04:35:14.406 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.406+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efd0406ace0 msgr2=0x7efd0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.406+0000 7efd1e904700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efd0406ace0 0x7efd0406d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efd08006fd0 tx=0x7efd08008040 comp rx=0 tx=0).stop 2023-06-09T04:35:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.406+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180aed50 msgr2=0x7efd181f1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.406+0000 7efd1e904700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180aed50 0x7efd181f1e70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000c00 tx=0x7efd0c004750 comp rx=0 tx=0).stop 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.406+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 shutdown_connections 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.407+0000 7efd1e904700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efd0406ace0 0x7efd0406d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.407+0000 7efd1e904700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd1815d470 0x7efd18157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.407+0000 7efd1e904700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd180aed50 0x7efd181f1e70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.407+0000 7efd1e904700 1 --2- 172.21.15.92:0/813233485 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd180ce3d0 0x7efd180c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.407+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 >> 172.21.15.92:0/813233485 conn(0x7efd180cc380 msgr2=0x7efd18158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.407+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 shutdown_connections 2023-06-09T04:35:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.407+0000 7efd1e904700 1 -- 172.21.15.92:0/813233485 wait complete. 2023-06-09T04:35:14.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-09T04:35:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.538+0000 7f1435d77700 1 Processor -- start 2023-06-09T04:35:14.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.538+0000 7f1435d77700 1 -- start start 2023-06-09T04:35:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f1435d77700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 0x7f14300ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f1435d77700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14300ce730 con 0x7f14300cf010 2023-06-09T04:35:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f1435d77700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143006aaf0 con 0x7f14300ac400 2023-06-09T04:35:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f1435d77700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143015cba0 con 0x7f143015c7d0 2023-06-09T04:35:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f14300cf010 0x7f143006e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35740/0 (socket says 172.21.15.92:35740) 2023-06-09T04:35:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142ffff700 1 -- 172.21.15.92:0/3748105961 learned_addr learned my addr 172.21.15.92:0/3748105961 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142f7fe700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 0x7f14300ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411286535 0 0) 0x7f1428001490 con 0x7f14300cf010 2023-06-09T04:35:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141c0035b0 con 0x7f14300cf010 2023-06-09T04:35:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63771690 0 0) 0x7f141c0035b0 con 0x7f14300cf010 2023-06-09T04:35:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 msgr2=0x7f14300ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 0x7f14300ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 >> v1:172.21.15.123:6789/0 conn(0x7f14300ac400 legacy=0x7f14300ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1430069ee0 con 0x7f14300cf010 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f1428001b80 con 0x7f14300cf010 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 >> v1:172.21.15.92:6789/0 conn(0x7f14300cf010 legacy=0x7f143006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c003a30 0x7f141c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.539+0000 7f142e7fc700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f141c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142e7fc700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 0x7f141c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143015cba0 con 0x7f141c004850 2023-06-09T04:35:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c00bb10 con 0x7f141c003a30 2023-06-09T04:35:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c00bcd0 con 0x7f143015c7d0 2023-06-09T04:35:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142effd700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f141c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142f7fe700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c003a30 0x7f141c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142ffff700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 0x7f141c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142effd700 1 -- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 msgr2=0x7f141c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:14.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142effd700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f143015c7d0 0x7f141c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142effd700 1 -- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c003a30 msgr2=0x7f141c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142effd700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f141c003a30 0x7f141c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142effd700 1 -- 172.21.15.92:0/3748105961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0035b0 con 0x7f141c004850 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142effd700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f141c004cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1420001800 tx=0x7f141c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1216a134cf46d9d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14200012f0 con 0x7f141c004850 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14200052f0 con 0x7f141c004850 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.540+0000 7f142e7fc700 1 -- 172.21.15.92:0/3748105961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420006860 con 0x7f141c004850 2023-06-09T04:35:14.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.541+0000 7f1435d77700 1 -- 172.21.15.92:0/3748105961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 msgr2=0x7f141c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.541+0000 7f1435d77700 1 --2- 172.21.15.92:0/3748105961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f141c004cc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1420001800 tx=0x7f141c005880 comp rx=0 tx=0).stop 2023-06-09T04:35:14.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.541+0000 7f142ffff700 1 -- 172.21.15.92:0/3748105961 reap_dead start 2023-06-09T04:35:14.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.541+0000 7f1435d77700 1 -- 172.21.15.92:0/3748105961 shutdown_connections 2023-06-09T04:35:14.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.541+0000 7f1435d77700 1 -- 172.21.15.92:0/3748105961 >> 172.21.15.92:0/3748105961 conn(0x7f1430098c10 msgr2=0x7f14300cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:14.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.541+0000 7f1435d77700 1 -- 172.21.15.92:0/3748105961 shutdown_connections 2023-06-09T04:35:14.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.541+0000 7f1435d77700 1 -- 172.21.15.92:0/3748105961 wait complete. 2023-06-09T04:35:14.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.542+0000 7f1435d77700 1 Processor -- start 2023-06-09T04:35:14.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.542+0000 7f1435d77700 1 -- start start 2023-06-09T04:35:14.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1435d77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f14300c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1435d77700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14300c5ee0 0x7f1430156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1435d77700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14300ca890 0x7f14301f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1435d77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14300c3d50 con 0x7f141c004850 2023-06-09T04:35:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1435d77700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14300c3420 con 0x7f14300c5ee0 2023-06-09T04:35:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1435d77700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14300c3700 con 0x7f14300ca890 2023-06-09T04:35:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f14300c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f14300c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37416/0 (socket says 172.21.15.92:37416) 2023-06-09T04:35:14.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 -- 172.21.15.92:0/2036780464 learned_addr learned my addr 172.21.15.92:0/2036780464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 -- 172.21.15.92:0/2036780464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14300ca890 msgr2=0x7f14301f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14300ca890 0x7f14301f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 -- 172.21.15.92:0/2036780464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14300c5ee0 msgr2=0x7f1430156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14300c5ee0 0x7f1430156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 -- 172.21.15.92:0/2036780464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300c39e0 con 0x7f141c004850 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f142f7fe700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f14300c75a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1418009fd0 tx=0x7f1418011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1427fff700 1 -- 172.21.15.92:0/2036780464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418009610 con 0x7f141c004850 2023-06-09T04:35:14.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1427fff700 1 -- 172.21.15.92:0/2036780464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1418005c30 con 0x7f141c004850 2023-06-09T04:35:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.543+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14300cb0e0 con 0x7f141c004850 2023-06-09T04:35:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.544+0000 7f1427fff700 1 -- 172.21.15.92:0/2036780464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141800f6b0 con 0x7f141c004850 2023-06-09T04:35:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.544+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14301f55e0 con 0x7f141c004850 2023-06-09T04:35:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.544+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14300c4030 con 0x7f141c004850 2023-06-09T04:35:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.545+0000 7f1427fff700 1 -- 172.21.15.92:0/2036780464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f141800f810 con 0x7f141c004850 2023-06-09T04:35:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.545+0000 7f1427fff700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f141006ace0 0x7f141006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.545+0000 7f1427fff700 1 -- 172.21.15.92:0/2036780464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f14180899d0 con 0x7f141c004850 2023-06-09T04:35:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.545+0000 7f142effd700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f141006ace0 0x7f141006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.546+0000 7f142effd700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f141006ace0 0x7f141006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14301f5000 tx=0x7f1420011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.550+0000 7f1427fff700 1 -- 172.21.15.92:0/2036780464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f14180589b0 con 0x7f141c004850 2023-06-09T04:35:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.741+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f14300740e0 con 0x7f141c004850 2023-06-09T04:35:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.741+0000 7f1427fff700 1 -- 172.21.15.92:0/2036780464 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f1418058540 con 0x7f141c004850 2023-06-09T04:35:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:14.743 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-06-09T04:35:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:35:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-06-09T04:35:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-06-09T04:35:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:35:14.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:35:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-06-09T04:35:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-06-09T04:35:14.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:35:14.744 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-06-09T04:35:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f141006ace0 msgr2=0x7f141006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f141006ace0 0x7f141006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14301f5000 tx=0x7f1420011000 comp rx=0 tx=0).stop 2023-06-09T04:35:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 msgr2=0x7f14300c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f14300c75a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1418009fd0 tx=0x7f1418011d30 comp rx=0 tx=0).stop 2023-06-09T04:35:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 shutdown_connections 2023-06-09T04:35:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f141006ace0 0x7f141006d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14300ca890 0x7f14301f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f141c004850 0x7f14300c75a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 --2- 172.21.15.92:0/2036780464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14300c5ee0 0x7f1430156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 >> 172.21.15.92:0/2036780464 conn(0x7f1430098c10 msgr2=0x7f14300ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:14.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 shutdown_connections 2023-06-09T04:35:14.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.743+0000 7f1435d77700 1 -- 172.21.15.92:0/2036780464 wait complete. 2023-06-09T04:35:14.756 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-09T04:35:14.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-09T04:35:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.878+0000 7f3a7803b700 1 Processor -- start 2023-06-09T04:35:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.879+0000 7f3a7803b700 1 -- start start 2023-06-09T04:35:14.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.879+0000 7f3a7803b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a70069db0 0x7f3a7015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.879+0000 7f3a7803b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a700cf440 0x7f3a7006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.879+0000 7f3a7803b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7015d040 con 0x7f3a700cce60 2023-06-09T04:35:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.879+0000 7f3a7803b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a70160ab0 con 0x7f3a70069db0 2023-06-09T04:35:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.879+0000 7f3a7803b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7015ca10 con 0x7f3a700cf440 2023-06-09T04:35:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.879+0000 7f3a75dd7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a70069db0 0x7f3a7015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a75dd7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a70069db0 0x7f3a7015d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37424/0 (socket says 172.21.15.92:37424) 2023-06-09T04:35:14.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a75dd7700 1 -- 172.21.15.92:0/1441224003 learned_addr learned my addr 172.21.15.92:0/1441224003 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a765d8700 1 --1- 172.21.15.92:0/1441224003 >> v1:172.21.15.92:6789/0 conn(0x7f3a700cce60 0x7f3a7006ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35758/0 (socket says 172.21.15.92:35758) 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a755d6700 1 --2- 172.21.15.92:0/1441224003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a700cf440 0x7f3a7006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a75dd7700 1 -- 172.21.15.92:0/1441224003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a700cf440 msgr2=0x7f3a7006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a75dd7700 1 --2- 172.21.15.92:0/1441224003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a700cf440 0x7f3a7006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a75dd7700 1 -- 172.21.15.92:0/1441224003 >> v1:172.21.15.92:6789/0 conn(0x7f3a700cce60 legacy=0x7f3a7006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a75dd7700 1 -- 172.21.15.92:0/1441224003 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7006a6c0 con 0x7f3a70069db0 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a74dd5700 1 -- 172.21.15.92:0/1441224003 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3135998120 0 0) 0x7f3a7015d040 con 0x7f3a700cce60 2023-06-09T04:35:14.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a75dd7700 1 --2- 172.21.15.92:0/1441224003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a70069db0 0x7f3a7015d4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3a600098f0 tx=0x7f3a60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a41392e771fbcb6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a74dd5700 1 -- 172.21.15.92:0/1441224003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60011830 con 0x7f3a70069db0 2023-06-09T04:35:14.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a74dd5700 1 -- 172.21.15.92:0/1441224003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60011990 con 0x7f3a70069db0 2023-06-09T04:35:14.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.880+0000 7f3a74dd5700 1 -- 172.21.15.92:0/1441224003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60011d20 con 0x7f3a70069db0 2023-06-09T04:35:14.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 -- 172.21.15.92:0/1441224003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a70069db0 msgr2=0x7f3a7015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 --2- 172.21.15.92:0/1441224003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a70069db0 0x7f3a7015d4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3a600098f0 tx=0x7f3a60009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:14.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 -- 172.21.15.92:0/1441224003 shutdown_connections 2023-06-09T04:35:14.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 --2- 172.21.15.92:0/1441224003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a70069db0 0x7f3a7015d4d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 --2- 172.21.15.92:0/1441224003 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a700cf440 0x7f3a7006a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 -- 172.21.15.92:0/1441224003 >> 172.21.15.92:0/1441224003 conn(0x7f3a700cc380 msgr2=0x7f3a700cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 -- 172.21.15.92:0/1441224003 shutdown_connections 2023-06-09T04:35:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.881+0000 7f3a7803b700 1 -- 172.21.15.92:0/1441224003 wait complete. 2023-06-09T04:35:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.882+0000 7f3a7803b700 1 Processor -- start 2023-06-09T04:35:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.882+0000 7f3a7803b700 1 -- start start 2023-06-09T04:35:14.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a7803b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a70069db0 0x7f3a701e6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a7803b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a700cce60 0x7f3a700b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a7803b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a700cf440 0x7f3a7014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a7803b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a701e78b0 con 0x7f3a700cce60 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a7803b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a700b2210 con 0x7f3a700cf440 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a7803b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a701e6d30 con 0x7f3a70069db0 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a700cce60 0x7f3a700b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a700cce60 0x7f3a700b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37430/0 (socket says 172.21.15.92:37430) 2023-06-09T04:35:14.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 -- 172.21.15.92:0/3953618813 learned_addr learned my addr 172.21.15.92:0/3953618813 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a765d8700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a700cf440 0x7f3a7014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a75dd7700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a70069db0 0x7f3a701e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 -- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a70069db0 msgr2=0x7f3a701e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a70069db0 0x7f3a701e6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 -- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a700cf440 msgr2=0x7f3a7014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a700cf440 0x7f3a7014fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:14.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 -- 172.21.15.92:0/3953618813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60009550 con 0x7f3a700cce60 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a755d6700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a700cce60 0x7f3a700b1cd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00e870 tx=0x7f3a6c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a667fc700 1 -- 172.21.15.92:0/3953618813 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c009590 con 0x7f3a700cce60 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a667fc700 1 -- 172.21.15.92:0/3953618813 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c00ee70 con 0x7f3a700cce60 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a667fc700 1 -- 172.21.15.92:0/3953618813 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00ca50 con 0x7f3a700cce60 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.883+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a701e7010 con 0x7f3a700cce60 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.884+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a701e75d0 con 0x7f3a700cce60 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.884+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7006be50 con 0x7f3a700cce60 2023-06-09T04:35:14.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.885+0000 7f3a667fc700 1 -- 172.21.15.92:0/3953618813 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a6c00cbb0 con 0x7f3a700cce60 2023-06-09T04:35:14.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.885+0000 7f3a667fc700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a5406ace0 0x7f3a5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:14.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.885+0000 7f3a667fc700 1 -- 172.21.15.92:0/3953618813 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3a6c089110 con 0x7f3a700cce60 2023-06-09T04:35:14.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.885+0000 7f3a75dd7700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a5406ace0 0x7f3a5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:14.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.886+0000 7f3a75dd7700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a5406ace0 0x7f3a5406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009c40 tx=0x7f3a60008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:14.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:14.890+0000 7f3a667fc700 1 -- 172.21.15.92:0/3953618813 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a6c057f90 con 0x7f3a700cce60 2023-06-09T04:35:15.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:15.082+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3a7004e8e0 con 0x7f3a700cce60 2023-06-09T04:35:15.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:35:15.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:35:16.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.009+0000 7f3a667fc700 1 -- 172.21.15.92:0/3953618813 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3a6c00ce60 con 0x7f3a700cce60 2023-06-09T04:35:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a5406ace0 msgr2=0x7f3a5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a5406ace0 0x7f3a5406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009c40 tx=0x7f3a60008030 comp rx=0 tx=0).stop 2023-06-09T04:35:16.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a700cce60 msgr2=0x7f3a700b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a700cce60 0x7f3a700b1cd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00e870 tx=0x7f3a6c00ec30 comp rx=0 tx=0).stop 2023-06-09T04:35:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 shutdown_connections 2023-06-09T04:35:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a5406ace0 0x7f3a5406d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a700cf440 0x7f3a7014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a70069db0 0x7f3a701e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 --2- 172.21.15.92:0/3953618813 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a700cce60 0x7f3a700b1cd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 >> 172.21.15.92:0/3953618813 conn(0x7f3a700cc380 msgr2=0x7f3a700c9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.015+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 shutdown_connections 2023-06-09T04:35:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.016+0000 7f3a7803b700 1 -- 172.21.15.92:0/3953618813 wait complete. 2023-06-09T04:35:16.018 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-06-09T04:35:16.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-09T04:35:16.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.163+0000 7f3eb39ab700 1 Processor -- start 2023-06-09T04:35:16.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb39ab700 1 -- start start 2023-06-09T04:35:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb39ab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3eac158b00 0x7f3eac0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb39ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 0x7f3eac0cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb39ab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac15c590 con 0x7f3eac0b15a0 2023-06-09T04:35:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb39ab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac15d460 con 0x7f3eac158b00 2023-06-09T04:35:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb39ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac158ed0 con 0x7f3eac0cb550 2023-06-09T04:35:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb1f48700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3eac0b15a0 0x7f3eac06cbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35768/0 (socket says 172.21.15.92:35768) 2023-06-09T04:35:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.164+0000 7f3eb1f48700 1 -- 172.21.15.92:0/2316002143 learned_addr learned my addr 172.21.15.92:0/2316002143 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 0x7f3eac0cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb1747700 1 --2- 172.21.15.92:0/2316002143 >> v2:172.21.15.123:3300/0 conn(0x7f3eac158b00 0x7f3eac0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710479186 0 0) 0x7f3eac15c590 con 0x7f3eac0b15a0 2023-06-09T04:35:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e980057c0 con 0x7f3eac0b15a0 2023-06-09T04:35:16.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2899885936 0 0) 0x7f3e980057c0 con 0x7f3eac0b15a0 2023-06-09T04:35:16.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 msgr2=0x7f3eac0cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:16.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 0x7f3eac0cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 >> v2:172.21.15.123:3300/0 conn(0x7f3eac158b00 msgr2=0x7f3eac0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:16.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/2316002143 >> v2:172.21.15.123:3300/0 conn(0x7f3eac158b00 0x7f3eac0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac069db0 con 0x7f3eac0b15a0 2023-06-09T04:35:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f3ea40019f0 con 0x7f3eac0b15a0 2023-06-09T04:35:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 >> v1:172.21.15.92:6789/0 conn(0x7f3eac0b15a0 legacy=0x7f3eac06cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98005c40 0x7f3e98006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e98006a60 0x7f3e98006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 0x7f3e98007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac15d460 con 0x7f3e98006a60 2023-06-09T04:35:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac158ed0 con 0x7f3e98005c40 2023-06-09T04:35:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9800dc50 con 0x7f3eac0cb550 2023-06-09T04:35:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e98006a60 0x7f3e98006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb1f48700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 0x7f3e98007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb1747700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98005c40 0x7f3e98006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb0f46700 1 -- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 msgr2=0x7f3e98007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3eac0cb550 0x7f3e98007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb0f46700 1 -- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98005c40 msgr2=0x7f3e98006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e98005c40 0x7f3e98006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.165+0000 7f3eb0f46700 1 -- 172.21.15.92:0/2316002143 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980057c0 con 0x7f3e98006a60 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e98006a60 0x7f3e98006ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009b60 tx=0x7f3e98007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d79ae27894c56de server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800e660 con 0x7f3e98006a60 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea800e7c0 con 0x7f3e98006a60 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3ea3fff700 1 -- 172.21.15.92:0/2316002143 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800eb30 con 0x7f3e98006a60 2023-06-09T04:35:16.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2316002143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e98006a60 msgr2=0x7f3e98006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb39ab700 1 --2- 172.21.15.92:0/2316002143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e98006a60 0x7f3e98006ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009b60 tx=0x7f3e98007a90 comp rx=0 tx=0).stop 2023-06-09T04:35:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb1f48700 1 -- 172.21.15.92:0/2316002143 reap_dead start 2023-06-09T04:35:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2316002143 shutdown_connections 2023-06-09T04:35:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2316002143 >> 172.21.15.92:0/2316002143 conn(0x7f3eac0cfc40 msgr2=0x7f3eac0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2316002143 shutdown_connections 2023-06-09T04:35:16.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.166+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2316002143 wait complete. 2023-06-09T04:35:16.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.167+0000 7f3eb39ab700 1 Processor -- start 2023-06-09T04:35:16.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb39ab700 1 -- start start 2023-06-09T04:35:16.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb39ab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e98006a60 0x7f3eac1f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb39ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eac0ae260 0x7f3eac0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb39ab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3eac0c6a10 0x7f3eac153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb39ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0c8e80 con 0x7f3eac0ae260 2023-06-09T04:35:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb39ab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0b1fa0 con 0x7f3eac0c6a10 2023-06-09T04:35:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb39ab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0ae6d0 con 0x7f3e98006a60 2023-06-09T04:35:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb0f46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eac0ae260 0x7f3eac0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb0f46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eac0ae260 0x7f3eac0c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37448/0 (socket says 172.21.15.92:37448) 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb0f46700 1 -- 172.21.15.92:0/2050184974 learned_addr learned my addr 172.21.15.92:0/2050184974 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb1747700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e98006a60 0x7f3eac1f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.168+0000 7f3eb1f48700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3eac0c6a10 0x7f3eac153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb0f46700 1 -- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e98006a60 msgr2=0x7f3eac1f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e98006a60 0x7f3eac1f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb0f46700 1 -- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3eac0c6a10 msgr2=0x7f3eac153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3eac0c6a10 0x7f3eac153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb0f46700 1 -- 172.21.15.92:0/2050184974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac1f5c00 con 0x7f3eac0ae260 2023-06-09T04:35:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb1f48700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3eac0c6a10 0x7f3eac153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb0f46700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eac0ae260 0x7f3eac0c8940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e98006980 tx=0x7f3ea80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3ea1ffb700 1 -- 172.21.15.92:0/2050184974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8021070 con 0x7f3eac0ae260 2023-06-09T04:35:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3ea1ffb700 1 -- 172.21.15.92:0/2050184974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea80045d0 con 0x7f3eac0ae260 2023-06-09T04:35:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1f5ee0 con 0x7f3eac0ae260 2023-06-09T04:35:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3ea1ffb700 1 -- 172.21.15.92:0/2050184974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800e660 con 0x7f3eac0ae260 2023-06-09T04:35:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.169+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac0c6eb0 con 0x7f3eac0ae260 2023-06-09T04:35:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.170+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac1f6400 con 0x7f3eac0ae260 2023-06-09T04:35:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.171+0000 7f3ea1ffb700 1 -- 172.21.15.92:0/2050184974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ea8025070 con 0x7f3eac0ae260 2023-06-09T04:35:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.171+0000 7f3ea1ffb700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.171+0000 7f3ea1ffb700 1 -- 172.21.15.92:0/2050184974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3ea808a2b0 con 0x7f3eac0ae260 2023-06-09T04:35:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.171+0000 7f3eb1747700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:16.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.171+0000 7f3eb1747700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3e9006ace0 0x7f3e9006d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009c60 tx=0x7f3e9c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:16.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.176+0000 7f3ea1ffb700 1 -- 172.21.15.92:0/2050184974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3ea80556f0 con 0x7f3eac0ae260 2023-06-09T04:35:16.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:16.363+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3eac04e8e0 con 0x7f3eac0ae260 2023-06-09T04:35:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.011+0000 7f3ea1ffb700 1 -- 172.21.15.92:0/2050184974 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v22) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3ea8058d10 con 0x7f3eac0ae260 2023-06-09T04:35:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3e9006ace0 msgr2=0x7f3e9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3e9006ace0 0x7f3e9006d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009c60 tx=0x7f3e9c009210 comp rx=0 tx=0).stop 2023-06-09T04:35:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eac0ae260 msgr2=0x7f3eac0c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eac0ae260 0x7f3eac0c8940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e98006980 tx=0x7f3ea80047c0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 shutdown_connections 2023-06-09T04:35:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3e9006ace0 0x7f3e9006d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3eac0c6a10 0x7f3eac153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e98006a60 0x7f3eac1f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 --2- 172.21.15.92:0/2050184974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eac0ae260 0x7f3eac0c8940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.016+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 >> 172.21.15.92:0/2050184974 conn(0x7f3eac0cfc40 msgr2=0x7f3eac0aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.017+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 shutdown_connections 2023-06-09T04:35:17.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.017+0000 7f3eb39ab700 1 -- 172.21.15.92:0/2050184974 wait complete. 2023-06-09T04:35:17.027 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-06-09T04:35:17.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-09T04:35:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.168+0000 7ffb51fef700 1 Processor -- start 2023-06-09T04:35:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.168+0000 7ffb51fef700 1 -- start start 2023-06-09T04:35:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb51fef700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb4c0ce3d0 0x7ffb4c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb51fef700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb51fef700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4c0ce190 con 0x7ffb4c15d470 2023-06-09T04:35:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb51fef700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c072350 con 0x7ffb4c0ce3d0 2023-06-09T04:35:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb51fef700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c0cece0 con 0x7ffb4c0aed50 2023-06-09T04:35:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb4b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb4c0ce3d0 0x7ffb4c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb4b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb4c0ce3d0 0x7ffb4c0713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37454/0 (socket says 172.21.15.92:37454) 2023-06-09T04:35:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb4b7fe700 1 -- 172.21.15.92:0/107079998 learned_addr learned my addr 172.21.15.92:0/107079998 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.169+0000 7ffb4affd700 1 --2- 172.21.15.92:0/107079998 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4b7fe700 1 -- 172.21.15.92:0/107079998 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 msgr2=0x7ffb4c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4b7fe700 1 --2- 172.21.15.92:0/107079998 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4b7fe700 1 -- 172.21.15.92:0/107079998 >> v1:172.21.15.123:6789/0 conn(0x7ffb4c15d470 legacy=0x7ffb4c1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4b7fe700 1 -- 172.21.15.92:0/107079998 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c071ab0 con 0x7ffb4c0ce3d0 2023-06-09T04:35:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4b7fe700 1 --2- 172.21.15.92:0/107079998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c0713b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb340098f0 tx=0x7ffb34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773a0b7a2aa8c374 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4a7fc700 1 -- 172.21.15.92:0/107079998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb34011830 con 0x7ffb4c0ce3d0 2023-06-09T04:35:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4a7fc700 1 -- 172.21.15.92:0/107079998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb34011990 con 0x7ffb4c0ce3d0 2023-06-09T04:35:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb4a7fc700 1 -- 172.21.15.92:0/107079998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb34011d20 con 0x7ffb4c0ce3d0 2023-06-09T04:35:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb51fef700 1 -- 172.21.15.92:0/107079998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c0ce3d0 msgr2=0x7ffb4c0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb51fef700 1 --2- 172.21.15.92:0/107079998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c0713b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb340098f0 tx=0x7ffb34009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb51fef700 1 -- 172.21.15.92:0/107079998 shutdown_connections 2023-06-09T04:35:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb51fef700 1 --2- 172.21.15.92:0/107079998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c0713b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb51fef700 1 --2- 172.21.15.92:0/107079998 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.170+0000 7ffb51fef700 1 -- 172.21.15.92:0/107079998 >> 172.21.15.92:0/107079998 conn(0x7ffb4c0cc380 msgr2=0x7ffb4c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.171+0000 7ffb51fef700 1 -- 172.21.15.92:0/107079998 shutdown_connections 2023-06-09T04:35:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.171+0000 7ffb51fef700 1 -- 172.21.15.92:0/107079998 wait complete. 2023-06-09T04:35:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.172+0000 7ffb51fef700 1 Processor -- start 2023-06-09T04:35:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.172+0000 7ffb51fef700 1 -- start start 2023-06-09T04:35:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb51fef700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb51fef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb51fef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c15d470 0x7ffb4c1510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb51fef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c149ba0 con 0x7ffb4c15d470 2023-06-09T04:35:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb51fef700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c14d250 con 0x7ffb4c0ce3d0 2023-06-09T04:35:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb51fef700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c14d4e0 con 0x7ffb4c0aed50 2023-06-09T04:35:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c15d470 0x7ffb4c1510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c15d470 0x7ffb4c1510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37464/0 (socket says 172.21.15.92:37464) 2023-06-09T04:35:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb4bfff700 1 -- 172.21.15.92:0/1885969252 learned_addr learned my addr 172.21.15.92:0/1885969252 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:17.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb4b7fe700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb4affd700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.173+0000 7ffb4bfff700 1 -- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 msgr2=0x7ffb4c14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb4bfff700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c14e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb4bfff700 1 -- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb4c0ce3d0 msgr2=0x7ffb4c149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb4bfff700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c149660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb4bfff700 1 -- 172.21.15.92:0/1885969252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34009550 con 0x7ffb4c15d470 2023-06-09T04:35:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb4affd700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb4bfff700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c15d470 0x7ffb4c1510c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400cc50 tx=0x7ffb4400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb43fff700 1 -- 172.21.15.92:0/1885969252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400ad80 con 0x7ffb4c15d470 2023-06-09T04:35:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb43fff700 1 -- 172.21.15.92:0/1885969252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4400ce50 con 0x7ffb4c15d470 2023-06-09T04:35:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c14d7c0 con 0x7ffb4c15d470 2023-06-09T04:35:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb43fff700 1 -- 172.21.15.92:0/1885969252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb44003c40 con 0x7ffb4c15d470 2023-06-09T04:35:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.174+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c14dd80 con 0x7ffb4c15d470 2023-06-09T04:35:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.175+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c06be50 con 0x7ffb4c15d470 2023-06-09T04:35:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.175+0000 7ffb43fff700 1 -- 172.21.15.92:0/1885969252 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb440050d0 con 0x7ffb4c15d470 2023-06-09T04:35:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.176+0000 7ffb43fff700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffb2c06ace0 0x7ffb2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.176+0000 7ffb43fff700 1 -- 172.21.15.92:0/1885969252 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7ffb44089530 con 0x7ffb4c15d470 2023-06-09T04:35:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.176+0000 7ffb4b7fe700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffb2c06ace0 0x7ffb2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.176+0000 7ffb4b7fe700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffb2c06ace0 0x7ffb2c06d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb34000c00 tx=0x7ffb34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.180+0000 7ffb43fff700 1 -- 172.21.15.92:0/1885969252 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffb44054640 con 0x7ffb4c15d470 2023-06-09T04:35:17.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.410+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ffb4c04e8e0 con 0x7ffb4c15d470 2023-06-09T04:35:17.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.414+0000 7ffb43fff700 1 -- 172.21.15.92:0/1885969252 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ffb44057c60 con 0x7ffb4c15d470 2023-06-09T04:35:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffb2c06ace0 msgr2=0x7ffb2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffb2c06ace0 0x7ffb2c06d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb34000c00 tx=0x7ffb34008030 comp rx=0 tx=0).stop 2023-06-09T04:35:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c15d470 msgr2=0x7ffb4c1510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c15d470 0x7ffb4c1510c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400cc50 tx=0x7ffb4400a340 comp rx=0 tx=0).stop 2023-06-09T04:35:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 shutdown_connections 2023-06-09T04:35:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffb2c06ace0 0x7ffb2c06d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb4c15d470 0x7ffb4c1510c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb4c0aed50 0x7ffb4c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 --2- 172.21.15.92:0/1885969252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb4c0ce3d0 0x7ffb4c149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.418+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 >> 172.21.15.92:0/1885969252 conn(0x7ffb4c0cc380 msgr2=0x7ffb4c0cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.419+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 shutdown_connections 2023-06-09T04:35:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.419+0000 7ffb51fef700 1 -- 172.21.15.92:0/1885969252 wait complete. 2023-06-09T04:35:17.421 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 3 and data pool 4 2023-06-09T04:35:17.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:17.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.560+0000 7f9e4d884700 1 Processor -- start 2023-06-09T04:35:17.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.560+0000 7f9e4d884700 1 -- start start 2023-06-09T04:35:17.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e4d884700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e4815c4b0 0x7f9e480d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e4d884700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e480b45e0 0x7f9e480b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e4d884700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e480b4f70 con 0x7f9e4806b1c0 2023-06-09T04:35:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e4d884700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e480d1140 con 0x7f9e4815c4b0 2023-06-09T04:35:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e4d884700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4815c880 con 0x7f9e480b45e0 2023-06-09T04:35:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e46ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e4815c4b0 0x7f9e480d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e477fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9e4806b1c0 0x7f9e4806b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35804/0 (socket says 172.21.15.92:35804) 2023-06-09T04:35:17.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e46ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e4815c4b0 0x7f9e480d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37468/0 (socket says 172.21.15.92:37468) 2023-06-09T04:35:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e477fe700 1 -- 172.21.15.92:0/3690713297 learned_addr learned my addr 172.21.15.92:0/3690713297 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e467fc700 1 --2- 172.21.15.92:0/3690713297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e480b45e0 0x7f9e480b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3690713297 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546681829 0 0) 0x7f9e40001490 con 0x7f9e4806b1c0 2023-06-09T04:35:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3690713297 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e300035b0 con 0x7f9e4806b1c0 2023-06-09T04:35:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e46ffd700 1 -- 172.21.15.92:0/3690713297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e480b45e0 msgr2=0x7f9e480b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/3690713297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e480b45e0 0x7f9e480b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e46ffd700 1 -- 172.21.15.92:0/3690713297 >> v1:172.21.15.92:6789/0 conn(0x7f9e4806b1c0 legacy=0x7f9e4806b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e46ffd700 1 -- 172.21.15.92:0/3690713297 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480d1350 con 0x7f9e4815c4b0 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/3690713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e480d22f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009840 tx=0x7f9e3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518a5f5160c9eb3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3690713297 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c011830 con 0x7f9e4815c4b0 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3690713297 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c011990 con 0x7f9e4815c4b0 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.561+0000 7f9e45ffb700 1 -- 172.21.15.92:0/3690713297 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c011d20 con 0x7f9e4815c4b0 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 -- 172.21.15.92:0/3690713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 msgr2=0x7f9e480d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 --2- 172.21.15.92:0/3690713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e480d22f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009840 tx=0x7f9e3c009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:17.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 -- 172.21.15.92:0/3690713297 shutdown_connections 2023-06-09T04:35:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 --2- 172.21.15.92:0/3690713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e480d22f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 --2- 172.21.15.92:0/3690713297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e480b45e0 0x7f9e480b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 -- 172.21.15.92:0/3690713297 >> 172.21.15.92:0/3690713297 conn(0x7f9e480cf5e0 msgr2=0x7f9e480cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 -- 172.21.15.92:0/3690713297 shutdown_connections 2023-06-09T04:35:17.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.562+0000 7f9e4d884700 1 -- 172.21.15.92:0/3690713297 wait complete. 2023-06-09T04:35:17.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.563+0000 7f9e4d884700 1 Processor -- start 2023-06-09T04:35:17.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.563+0000 7f9e4d884700 1 -- start start 2023-06-09T04:35:17.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e4d884700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e4806b1c0 0x7f9e481f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e4d884700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e480b45e0 0x7f9e480b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e4d884700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e4806b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e4d884700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480b5e20 con 0x7f9e4815c4b0 2023-06-09T04:35:17.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e4d884700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480cc360 con 0x7f9e480b45e0 2023-06-09T04:35:17.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e4d884700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480cc640 con 0x7f9e4806b1c0 2023-06-09T04:35:17.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e4806b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e4806b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37478/0 (socket says 172.21.15.92:37478) 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 -- 172.21.15.92:0/1331313396 learned_addr learned my addr 172.21.15.92:0/1331313396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e4806b1c0 0x7f9e481f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e467fc700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e480b45e0 0x7f9e480b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 -- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e4806b1c0 msgr2=0x7f9e481f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e4806b1c0 0x7f9e481f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 -- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e480b45e0 msgr2=0x7f9e480b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e480b45e0 0x7f9e480b8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.564+0000 7f9e477fe700 1 -- 172.21.15.92:0/1331313396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0094a0 con 0x7f9e4815c4b0 2023-06-09T04:35:17.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.565+0000 7f9e477fe700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e4806b640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000cb40 tx=0x7f9e4000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.565+0000 7f9e2ffff700 1 -- 172.21.15.92:0/1331313396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000ab50 con 0x7f9e4815c4b0 2023-06-09T04:35:17.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.565+0000 7f9e2ffff700 1 -- 172.21.15.92:0/1331313396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000acb0 con 0x7f9e4815c4b0 2023-06-09T04:35:17.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.565+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e480cc920 con 0x7f9e4815c4b0 2023-06-09T04:35:17.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.565+0000 7f9e2ffff700 1 -- 172.21.15.92:0/1331313396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40012620 con 0x7f9e4815c4b0 2023-06-09T04:35:17.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.565+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e481f2050 con 0x7f9e4815c4b0 2023-06-09T04:35:17.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.566+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e480ccb60 con 0x7f9e4815c4b0 2023-06-09T04:35:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.566+0000 7f9e2ffff700 1 -- 172.21.15.92:0/1331313396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e4001b3b0 con 0x7f9e4815c4b0 2023-06-09T04:35:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.566+0000 7f9e2ffff700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9e3406ace0 0x7f9e3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.567+0000 7f9e2ffff700 1 -- 172.21.15.92:0/1331313396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e40089040 con 0x7f9e4815c4b0 2023-06-09T04:35:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.567+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9e3406ace0 0x7f9e3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.567+0000 7f9e46ffd700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9e3406ace0 0x7f9e3406d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.571+0000 7f9e2ffff700 1 -- 172.21.15.92:0/1331313396 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e40054180 con 0x7f9e4815c4b0 2023-06-09T04:35:17.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.758+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e480b4f70 con 0x7f9e4815c4b0 2023-06-09T04:35:17.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.759+0000 7f9e2ffff700 1 -- 172.21.15.92:0/1331313396 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18268 (secure 0 0 0) 0x7f9e480b4f70 con 0x7f9e4815c4b0 2023-06-09T04:35:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:17.761 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:17.411861+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:17.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9e3406ace0 msgr2=0x7f9e3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9e3406ace0 0x7f9e3406d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c01c040 comp rx=0 tx=0).stop 2023-06-09T04:35:17.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 msgr2=0x7f9e4806b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e4806b640 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000cb40 tx=0x7f9e4000cf00 comp rx=0 tx=0).stop 2023-06-09T04:35:17.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 shutdown_connections 2023-06-09T04:35:17.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9e3406ace0 0x7f9e3406d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e4815c4b0 0x7f9e4806b640 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e4806b1c0 0x7f9e481f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 --2- 172.21.15.92:0/1331313396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e480b45e0 0x7f9e480b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 >> 172.21.15.92:0/1331313396 conn(0x7f9e480cf5e0 msgr2=0x7f9e480cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:17.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.761+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 shutdown_connections 2023-06-09T04:35:17.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.762+0000 7f9e4d884700 1 -- 172.21.15.92:0/1331313396 wait complete. 2023-06-09T04:35:17.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fac01cae700 1 Processor -- start 2023-06-09T04:35:17.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fac01cae700 1 -- start start 2023-06-09T04:35:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fac01cae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0b1360 0x7fabfc0b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fac01cae700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fabfc0aeaa0 0x7fabfc0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fac01cae700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc0adde0 con 0x7fabfc0ae630 2023-06-09T04:35:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fac01cae700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc0ae270 con 0x7fabfc0aeaa0 2023-06-09T04:35:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fac01cae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0af0f0 con 0x7fabfc0b1360 2023-06-09T04:35:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.894+0000 7fabf2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fabfc0ae630 0x7fabfc0ac400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35816/0 (socket says 172.21.15.92:35816) 2023-06-09T04:35:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabf2ffd700 1 -- 172.21.15.92:0/1275167895 learned_addr learned my addr 172.21.15.92:0/1275167895 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfb7fe700 1 --2- 172.21.15.92:0/1275167895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0b1360 0x7fabfc0b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfbfff700 1 --2- 172.21.15.92:0/1275167895 >> v2:172.21.15.123:3300/0 conn(0x7fabfc0aeaa0 0x7fabfc0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfaffd700 1 -- 172.21.15.92:0/1275167895 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099715780 0 0) 0x7fabfc0adde0 con 0x7fabfc0ae630 2023-06-09T04:35:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfaffd700 1 -- 172.21.15.92:0/1275167895 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe80035b0 con 0x7fabfc0ae630 2023-06-09T04:35:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfaffd700 1 -- 172.21.15.92:0/1275167895 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 604400242 0 0) 0x7fabe80035b0 con 0x7fabfc0ae630 2023-06-09T04:35:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfb7fe700 1 -- 172.21.15.92:0/1275167895 >> v2:172.21.15.123:3300/0 conn(0x7fabfc0aeaa0 msgr2=0x7fabfc0acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfb7fe700 1 --2- 172.21.15.92:0/1275167895 >> v2:172.21.15.123:3300/0 conn(0x7fabfc0aeaa0 0x7fabfc0acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfb7fe700 1 -- 172.21.15.92:0/1275167895 >> v1:172.21.15.92:6789/0 conn(0x7fabfc0ae630 legacy=0x7fabfc0ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfb7fe700 1 -- 172.21.15.92:0/1275167895 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc0b1c70 con 0x7fabfc0b1360 2023-06-09T04:35:17.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfb7fe700 1 --2- 172.21.15.92:0/1275167895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0b1360 0x7fabfc0b1730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabec009820 tx=0x7fabec009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c4936f8c44cc4a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfaffd700 1 -- 172.21.15.92:0/1275167895 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4001670 con 0x7fabfc0b1360 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfaffd700 1 -- 172.21.15.92:0/1275167895 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec005c30 con 0x7fabfc0b1360 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.895+0000 7fabfaffd700 1 -- 172.21.15.92:0/1275167895 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec004030 con 0x7fabfc0b1360 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 -- 172.21.15.92:0/1275167895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0b1360 msgr2=0x7fabfc0b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 --2- 172.21.15.92:0/1275167895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0b1360 0x7fabfc0b1730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fabec009820 tx=0x7fabec009b30 comp rx=0 tx=0).stop 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 -- 172.21.15.92:0/1275167895 shutdown_connections 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 --2- 172.21.15.92:0/1275167895 >> v2:172.21.15.123:3300/0 conn(0x7fabfc0aeaa0 0x7fabfc0acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 --2- 172.21.15.92:0/1275167895 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0b1360 0x7fabfc0b1730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 -- 172.21.15.92:0/1275167895 >> 172.21.15.92:0/1275167895 conn(0x7fabfc098c10 msgr2=0x7fabfc0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:17.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 -- 172.21.15.92:0/1275167895 shutdown_connections 2023-06-09T04:35:17.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.896+0000 7fac01cae700 1 -- 172.21.15.92:0/1275167895 wait complete. 2023-06-09T04:35:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.897+0000 7fac01cae700 1 Processor -- start 2023-06-09T04:35:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.897+0000 7fac01cae700 1 -- start start 2023-06-09T04:35:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fac01cae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabfc0ae630 0x7fabfc154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fac01cae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0aeaa0 0x7fabfc0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fac01cae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabfc0b1360 0x7fabfc0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fac01cae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0c8f00 con 0x7fabfc0b1360 2023-06-09T04:35:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fac01cae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0c91e0 con 0x7fabfc0ae630 2023-06-09T04:35:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fac01cae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0c94c0 con 0x7fabfc0aeaa0 2023-06-09T04:35:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabfc0b1360 0x7fabfc0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabfc0b1360 0x7fabfc0b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37492/0 (socket says 172.21.15.92:37492) 2023-06-09T04:35:17.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 -- 172.21.15.92:0/4053206459 learned_addr learned my addr 172.21.15.92:0/4053206459 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabf2ffd700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0aeaa0 0x7fabfc0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfb7fe700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabfc0ae630 0x7fabfc154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 -- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0aeaa0 msgr2=0x7fabfc0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0aeaa0 0x7fabfc0c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 -- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabfc0ae630 msgr2=0x7fabfc154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabfc0ae630 0x7fabfc154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfbfff700 1 -- 172.21.15.92:0/4053206459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec009430 con 0x7fabfc0b1360 2023-06-09T04:35:17.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.898+0000 7fabfb7fe700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabfc0ae630 0x7fabfc154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.899+0000 7fabfbfff700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabfc0b1360 0x7fabfc0b3760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabf400efc0 tx=0x7fabf400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.899+0000 7fabf8ff9700 1 -- 172.21.15.92:0/4053206459 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4009ea0 con 0x7fabfc0b1360 2023-06-09T04:35:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.899+0000 7fabf8ff9700 1 -- 172.21.15.92:0/4053206459 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf400f040 con 0x7fabfc0b1360 2023-06-09T04:35:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.899+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc0c97a0 con 0x7fabfc0b1360 2023-06-09T04:35:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.899+0000 7fabf8ff9700 1 -- 172.21.15.92:0/4053206459 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4014480 con 0x7fabfc0b1360 2023-06-09T04:35:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.899+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc1553f0 con 0x7fabfc0b1360 2023-06-09T04:35:17.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.900+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc0740e0 con 0x7fabfc0b1360 2023-06-09T04:35:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.900+0000 7fabf8ff9700 1 -- 172.21.15.92:0/4053206459 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabf40145e0 con 0x7fabfc0b1360 2023-06-09T04:35:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.901+0000 7fabf8ff9700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fabdc06ace0 0x7fabdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.901+0000 7fabfb7fe700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fabdc06ace0 0x7fabdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.901+0000 7fabf8ff9700 1 -- 172.21.15.92:0/4053206459 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fabf4089d40 con 0x7fabfc0b1360 2023-06-09T04:35:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.901+0000 7fabfb7fe700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fabdc06ace0 0x7fabdc06d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabec009b70 tx=0x7fabec008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:17.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:17.905+0000 7fabf8ff9700 1 -- 172.21.15.92:0/4053206459 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabf4054df0 con 0x7fabfc0b1360 2023-06-09T04:35:18.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.097+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabfc0c3160 con 0x7fabfc0b1360 2023-06-09T04:35:18.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.099+0000 7fabf8ff9700 1 -- 172.21.15.92:0/4053206459 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18268 (secure 0 0 0) 0x7fabf4058410 con 0x7fabfc0b1360 2023-06-09T04:35:18.100 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:18.100 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:17.411861+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:18.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fabdc06ace0 msgr2=0x7fabdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:18.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fabdc06ace0 0x7fabdc06d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabec009b70 tx=0x7fabec008070 comp rx=0 tx=0).stop 2023-06-09T04:35:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabfc0b1360 msgr2=0x7fabfc0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabfc0b1360 0x7fabfc0b3760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabf400efc0 tx=0x7fabf400c3a0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 shutdown_connections 2023-06-09T04:35:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fabdc06ace0 0x7fabdc06d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabfc0b1360 0x7fabfc0b3760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabfc0ae630 0x7fabfc154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 --2- 172.21.15.92:0/4053206459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fabfc0aeaa0 0x7fabfc0c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 >> 172.21.15.92:0/4053206459 conn(0x7fabfc098c10 msgr2=0x7fabfc06e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:18.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 shutdown_connections 2023-06-09T04:35:18.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.101+0000 7fac01cae700 1 -- 172.21.15.92:0/4053206459 wait complete. 2023-06-09T04:35:18.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-09T04:35:18.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.235+0000 7febe4780700 1 Processor -- start 2023-06-09T04:35:18.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.235+0000 7febe4780700 1 -- start start 2023-06-09T04:35:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe4780700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7febdc0cf440 0x7febdc15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe4780700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cce60 0x7febdc06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe4780700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febdc15d040 con 0x7febdc069db0 2023-06-09T04:35:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe4780700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febdc160ab0 con 0x7febdc0cf440 2023-06-09T04:35:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe4780700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc15ca10 con 0x7febdc0cce60 2023-06-09T04:35:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe2d1d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cce60 0x7febdc06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe1d1b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7febdc0cf440 0x7febdc15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe2d1d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cce60 0x7febdc06ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46406/0 (socket says 172.21.15.92:46406) 2023-06-09T04:35:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.236+0000 7febe2d1d700 1 -- 172.21.15.92:0/3808145065 learned_addr learned my addr 172.21.15.92:0/3808145065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe2d1d700 1 -- 172.21.15.92:0/3808145065 >> v2:172.21.15.123:3300/0 conn(0x7febdc0cf440 msgr2=0x7febdc15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe2d1d700 1 --2- 172.21.15.92:0/3808145065 >> v2:172.21.15.123:3300/0 conn(0x7febdc0cf440 0x7febdc15d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe2d1d700 1 -- 172.21.15.92:0/3808145065 >> v1:172.21.15.123:6789/0 conn(0x7febdc069db0 legacy=0x7febdc06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe2d1d700 1 -- 172.21.15.92:0/3808145065 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc160d50 con 0x7febdc0cce60 2023-06-09T04:35:18.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe1d1b700 1 --2- 172.21.15.92:0/3808145065 >> v2:172.21.15.123:3300/0 conn(0x7febdc0cf440 0x7febdc15d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe2d1d700 1 --2- 172.21.15.92:0/3808145065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cce60 0x7febdc06ee20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e50 tx=0x7febd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f170dc398b54deea server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe151a700 1 -- 172.21.15.92:0/3808145065 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400e070 con 0x7febdc0cce60 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe151a700 1 -- 172.21.15.92:0/3808145065 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd400b910 con 0x7febdc0cce60 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe151a700 1 -- 172.21.15.92:0/3808145065 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400bcc0 con 0x7febdc0cce60 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe4780700 1 -- 172.21.15.92:0/3808145065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cce60 msgr2=0x7febdc06ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe4780700 1 --2- 172.21.15.92:0/3808145065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cce60 0x7febdc06ee20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7febd4009e50 tx=0x7febd400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe4780700 1 -- 172.21.15.92:0/3808145065 shutdown_connections 2023-06-09T04:35:18.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe4780700 1 --2- 172.21.15.92:0/3808145065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cce60 0x7febdc06ee20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe4780700 1 --2- 172.21.15.92:0/3808145065 >> v2:172.21.15.123:3300/0 conn(0x7febdc0cf440 0x7febdc15d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.237+0000 7febe4780700 1 -- 172.21.15.92:0/3808145065 >> 172.21.15.92:0/3808145065 conn(0x7febdc0cc380 msgr2=0x7febdc0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:18.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.238+0000 7febe4780700 1 -- 172.21.15.92:0/3808145065 shutdown_connections 2023-06-09T04:35:18.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.238+0000 7febe4780700 1 -- 172.21.15.92:0/3808145065 wait complete. 2023-06-09T04:35:18.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 Processor -- start 2023-06-09T04:35:18.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 -- start start 2023-06-09T04:35:18.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febdc069db0 0x7febdc156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:18.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0cce60 0x7febdc155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:18.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cf440 0x7febdc0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:18.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0b3140 con 0x7febdc0cce60 2023-06-09T04:35:18.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0b3fa0 con 0x7febdc069db0 2023-06-09T04:35:18.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.239+0000 7febe4780700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0b3390 con 0x7febdc0cf440 2023-06-09T04:35:18.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0cce60 0x7febdc155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:18.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0cce60 0x7febdc155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37502/0 (socket says 172.21.15.92:37502) 2023-06-09T04:35:18.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 -- 172.21.15.92:0/3472831572 learned_addr learned my addr 172.21.15.92:0/3472831572 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe2d1d700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cf440 0x7febdc0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe251c700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febdc069db0 0x7febdc156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 -- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cf440 msgr2=0x7febdc0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cf440 0x7febdc0b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 -- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febdc069db0 msgr2=0x7febdc156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febdc069db0 0x7febdc156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:18.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 -- 172.21.15.92:0/3472831572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd80094a0 con 0x7febdc0cce60 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe1d1b700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0cce60 0x7febdc155a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7febd800ebe0 tx=0x7febd800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febd2ffd700 1 -- 172.21.15.92:0/3472831572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8009760 con 0x7febdc0cce60 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febd2ffd700 1 -- 172.21.15.92:0/3472831572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd800c850 con 0x7febdc0cce60 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd4009b30 con 0x7febdc0cce60 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febd2ffd700 1 -- 172.21.15.92:0/3472831572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800cbe0 con 0x7febdc0cce60 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.240+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc0b3950 con 0x7febdc0cce60 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.241+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc06be50 con 0x7febdc0cce60 2023-06-09T04:35:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.242+0000 7febd2ffd700 1 -- 172.21.15.92:0/3472831572 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febd800cd40 con 0x7febdc0cce60 2023-06-09T04:35:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.242+0000 7febd2ffd700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7febc006ace0 0x7febc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.242+0000 7febd2ffd700 1 -- 172.21.15.92:0/3472831572 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7febd8089440 con 0x7febdc0cce60 2023-06-09T04:35:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.242+0000 7febe251c700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7febc006ace0 0x7febc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.242+0000 7febe251c700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7febc006ace0 0x7febc006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febdc15c870 tx=0x7febcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.246+0000 7febd2ffd700 1 -- 172.21.15.92:0/3472831572 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7febd80544f0 con 0x7febdc0cce60 2023-06-09T04:35:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:18.480+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7febdc04e8e0 con 0x7febdc0cce60 2023-06-09T04:35:19.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.427+0000 7febd2ffd700 1 -- 172.21.15.92:0/3472831572 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7febd8057b10 con 0x7febdc0cce60 2023-06-09T04:35:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.429+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7febc006ace0 msgr2=0x7febc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.429+0000 7febe4780700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7febc006ace0 0x7febc006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febdc15c870 tx=0x7febcc009040 comp rx=0 tx=0).stop 2023-06-09T04:35:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0cce60 msgr2=0x7febdc155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0cce60 0x7febdc155a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7febd800ebe0 tx=0x7febd800eef0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 shutdown_connections 2023-06-09T04:35:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7febc006ace0 0x7febc006d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7febdc0cf440 0x7febdc0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febdc069db0 0x7febdc156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 --2- 172.21.15.92:0/3472831572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febdc0cce60 0x7febdc155a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 >> 172.21.15.92:0/3472831572 conn(0x7febdc0cc380 msgr2=0x7febdc0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 shutdown_connections 2023-06-09T04:35:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.430+0000 7febe4780700 1 -- 172.21.15.92:0/3472831572 wait complete. 2023-06-09T04:35:19.445 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-09T04:35:19.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-09T04:35:19.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.572+0000 7f54da55c700 1 Processor -- start 2023-06-09T04:35:19.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.572+0000 7f54da55c700 1 -- start start 2023-06-09T04:35:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54da55c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f54d40aed50 0x7f54d40713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54da55c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54d415d470 0x7f54d415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54da55c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d40ce190 con 0x7f54d40ce3d0 2023-06-09T04:35:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54da55c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d4072350 con 0x7f54d415d470 2023-06-09T04:35:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54da55c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d4071ab0 con 0x7f54d40aed50 2023-06-09T04:35:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54d415d470 0x7f54d415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:19.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54d415d470 0x7f54d415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37512/0 (socket says 172.21.15.92:37512) 2023-06-09T04:35:19.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 -- 172.21.15.92:0/3344276615 learned_addr learned my addr 172.21.15.92:0/3344276615 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:19.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d37fe700 1 --2- 172.21.15.92:0/3344276615 >> v2:172.21.15.123:3300/0 conn(0x7f54d40aed50 0x7f54d40713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:19.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 -- 172.21.15.92:0/3344276615 >> v2:172.21.15.123:3300/0 conn(0x7f54d40aed50 msgr2=0x7f54d40713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:19.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 --2- 172.21.15.92:0/3344276615 >> v2:172.21.15.123:3300/0 conn(0x7f54d40aed50 0x7f54d40713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 -- 172.21.15.92:0/3344276615 >> v1:172.21.15.123:6789/0 conn(0x7f54d40ce3d0 legacy=0x7f54d40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:19.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 -- 172.21.15.92:0/3344276615 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d4071d90 con 0x7f54d415d470 2023-06-09T04:35:19.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d8af9700 1 --2- 172.21.15.92:0/3344276615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d415d470 0x7f54d415d8b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54d40c33a0 tx=0x7f54cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db61e81b04165a53 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:19.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.573+0000 7f54d2ffd700 1 -- 172.21.15.92:0/3344276615 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00e070 con 0x7f54d415d470 2023-06-09T04:35:19.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54d2ffd700 1 -- 172.21.15.92:0/3344276615 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc003a20 con 0x7f54d415d470 2023-06-09T04:35:19.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54d2ffd700 1 -- 172.21.15.92:0/3344276615 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc003dd0 con 0x7f54d415d470 2023-06-09T04:35:19.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 -- 172.21.15.92:0/3344276615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d415d470 msgr2=0x7f54d415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:19.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 --2- 172.21.15.92:0/3344276615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d415d470 0x7f54d415d8b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f54d40c33a0 tx=0x7f54cc00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:19.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 -- 172.21.15.92:0/3344276615 shutdown_connections 2023-06-09T04:35:19.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 --2- 172.21.15.92:0/3344276615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d415d470 0x7f54d415d8b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 --2- 172.21.15.92:0/3344276615 >> v2:172.21.15.123:3300/0 conn(0x7f54d40aed50 0x7f54d40713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 -- 172.21.15.92:0/3344276615 >> 172.21.15.92:0/3344276615 conn(0x7f54d40cc380 msgr2=0x7f54d40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:19.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 -- 172.21.15.92:0/3344276615 shutdown_connections 2023-06-09T04:35:19.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.574+0000 7f54da55c700 1 -- 172.21.15.92:0/3344276615 wait complete. 2023-06-09T04:35:19.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.575+0000 7f54da55c700 1 Processor -- start 2023-06-09T04:35:19.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.575+0000 7f54da55c700 1 -- start start 2023-06-09T04:35:19.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54da55c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d40aed50 0x7f54d40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54da55c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d40ce3d0 0x7f54d40ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54da55c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54d415d470 0x7f54d4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54da55c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d414c6a0 con 0x7f54d40aed50 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54da55c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40ac6e0 con 0x7f54d40ce3d0 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54da55c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40ac980 con 0x7f54d415d470 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d40aed50 0x7f54d40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54d3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d40aed50 0x7f54d40c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37514/0 (socket says 172.21.15.92:37514) 2023-06-09T04:35:19.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54d3fff700 1 -- 172.21.15.92:0/1459381164 learned_addr learned my addr 172.21.15.92:0/1459381164 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:19.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54d37fe700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d40ce3d0 0x7f54d40ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:19.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.576+0000 7f54d8af9700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54d415d470 0x7f54d4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:19.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d3fff700 1 -- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54d415d470 msgr2=0x7f54d4069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:19.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d3fff700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54d415d470 0x7f54d4069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d3fff700 1 -- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d40ce3d0 msgr2=0x7f54d40ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:19.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d3fff700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d40ce3d0 0x7f54d40ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:19.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d3fff700 1 -- 172.21.15.92:0/1459381164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc009b30 con 0x7f54d40aed50 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d37fe700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d40ce3d0 0x7f54d40ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d3fff700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d40aed50 0x7f54d40c7780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c4d0 tx=0x7f54c800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d0ff9700 1 -- 172.21.15.92:0/1459381164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011790 con 0x7f54d40aed50 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d0ff9700 1 -- 172.21.15.92:0/1459381164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c80118f0 con 0x7f54d40aed50 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d40acc60 con 0x7f54d40aed50 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54d0ff9700 1 -- 172.21.15.92:0/1459381164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011ca0 con 0x7f54d40aed50 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.577+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d40ad220 con 0x7f54d40aed50 2023-06-09T04:35:19.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.578+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d41596c0 con 0x7f54d40aed50 2023-06-09T04:35:19.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.578+0000 7f54d0ff9700 1 -- 172.21.15.92:0/1459381164 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54c801c460 con 0x7f54d40aed50 2023-06-09T04:35:19.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.579+0000 7f54d0ff9700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f54c006ace0 0x7f54c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:19.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.579+0000 7f54d37fe700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f54c006ace0 0x7f54c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:19.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.579+0000 7f54d0ff9700 1 -- 172.21.15.92:0/1459381164 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f54c80893d0 con 0x7f54d40aed50 2023-06-09T04:35:19.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.579+0000 7f54d37fe700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f54c006ace0 0x7f54c006d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54d4157bc0 tx=0x7f54c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:19.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.583+0000 7f54d0ff9700 1 -- 172.21.15.92:0/1459381164 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f54d41596c0 con 0x7f54d40aed50 2023-06-09T04:35:19.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:19.815+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f54d41596c0 con 0x7f54d40aed50 2023-06-09T04:35:20.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.428+0000 7f54d0ff9700 1 -- 172.21.15.92:0/1459381164 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f54d41596c0 con 0x7f54d40aed50 2023-06-09T04:35:20.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f54c006ace0 msgr2=0x7f54c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f54c006ace0 0x7f54c006d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54d4157bc0 tx=0x7f54c4008040 comp rx=0 tx=0).stop 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d40aed50 msgr2=0x7f54d40c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d40aed50 0x7f54d40c7780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f54c800c4d0 tx=0x7f54c800c890 comp rx=0 tx=0).stop 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 shutdown_connections 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f54c006ace0 0x7f54c006d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54d415d470 0x7f54d4069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54d40aed50 0x7f54d40c7780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 --2- 172.21.15.92:0/1459381164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54d40ce3d0 0x7f54d40ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 >> 172.21.15.92:0/1459381164 conn(0x7f54d40cc380 msgr2=0x7f54d40b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 shutdown_connections 2023-06-09T04:35:20.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.432+0000 7f54da55c700 1 -- 172.21.15.92:0/1459381164 wait complete. 2023-06-09T04:35:20.445 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-09T04:35:20.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:35:20.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.564+0000 7fcf4118b700 1 Processor -- start 2023-06-09T04:35:20.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.564+0000 7fcf4118b700 1 -- start start 2023-06-09T04:35:20.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcf3c158cf0 0x7fcf3c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcf3c06d220 0x7fcf3c0cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c0cb540 con 0x7fcf3c158920 2023-06-09T04:35:20.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c06d7c0 con 0x7fcf3c158cf0 2023-06-09T04:35:20.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c159f30 con 0x7fcf3c06d220 2023-06-09T04:35:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcf3c158cf0 0x7fcf3c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcf3c158cf0 0x7fcf3c0cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37524/0 (socket says 172.21.15.92:37524) 2023-06-09T04:35:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 -- 172.21.15.92:0/3692478625 learned_addr learned my addr 172.21.15.92:0/3692478625 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 -- 172.21.15.92:0/3692478625 >> v2:172.21.15.123:3300/0 conn(0x7fcf3c06d220 msgr2=0x7fcf3c0cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3b59e700 1 --2- 172.21.15.92:0/3692478625 >> v2:172.21.15.123:3300/0 conn(0x7fcf3c06d220 0x7fcf3c0cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 --2- 172.21.15.92:0/3692478625 >> v2:172.21.15.123:3300/0 conn(0x7fcf3c06d220 0x7fcf3c0cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 -- 172.21.15.92:0/3692478625 >> v1:172.21.15.123:6789/0 conn(0x7fcf3c158920 legacy=0x7fcf3c15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:20.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 -- 172.21.15.92:0/3692478625 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c159170 con 0x7fcf3c158cf0 2023-06-09T04:35:20.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf3a59c700 1 --2- 172.21.15.92:0/3692478625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c158cf0 0x7fcf3c0cd1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c15c490 tx=0x7fcf2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb2f43e48ba19e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:20.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf39d9b700 1 -- 172.21.15.92:0/3692478625 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004230 con 0x7fcf3c158cf0 2023-06-09T04:35:20.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf39d9b700 1 -- 172.21.15.92:0/3692478625 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c004390 con 0x7fcf3c158cf0 2023-06-09T04:35:20.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf39d9b700 1 -- 172.21.15.92:0/3692478625 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0036a0 con 0x7fcf3c158cf0 2023-06-09T04:35:20.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- 172.21.15.92:0/3692478625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c158cf0 msgr2=0x7fcf3c0cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 --2- 172.21.15.92:0/3692478625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c158cf0 0x7fcf3c0cd1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c15c490 tx=0x7fcf2c009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- 172.21.15.92:0/3692478625 shutdown_connections 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 --2- 172.21.15.92:0/3692478625 >> v2:172.21.15.123:3300/0 conn(0x7fcf3c06d220 0x7fcf3c0cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 --2- 172.21.15.92:0/3692478625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c158cf0 0x7fcf3c0cd1e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- 172.21.15.92:0/3692478625 >> 172.21.15.92:0/3692478625 conn(0x7fcf3c0b1800 msgr2=0x7fcf3c06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- 172.21.15.92:0/3692478625 shutdown_connections 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.566+0000 7fcf4118b700 1 -- 172.21.15.92:0/3692478625 wait complete. 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 Processor -- start 2023-06-09T04:35:20.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 -- start start 2023-06-09T04:35:20.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c06d220 0x7fcf3c15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c158920 0x7fcf3c14f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf3c1638e0 0x7fcf3c163d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c15c060 con 0x7fcf3c06d220 2023-06-09T04:35:20.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c1642a0 con 0x7fcf3c158920 2023-06-09T04:35:20.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c152fd0 con 0x7fcf3c1638e0 2023-06-09T04:35:20.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c06d220 0x7fcf3c15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c06d220 0x7fcf3c15bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37528/0 (socket says 172.21.15.92:37528) 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 -- 172.21.15.92:0/4025634858 learned_addr learned my addr 172.21.15.92:0/4025634858 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3a59c700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c158920 0x7fcf3c14f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3b59e700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf3c1638e0 0x7fcf3c163d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 -- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf3c1638e0 msgr2=0x7fcf3c163d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf3c1638e0 0x7fcf3c163d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 -- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c158920 msgr2=0x7fcf3c14f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c158920 0x7fcf3c14f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 -- 172.21.15.92:0/4025634858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c009550 con 0x7fcf3c06d220 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf3ad9d700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c06d220 0x7fcf3c15bb20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c4a0 tx=0x7fcf3000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf237fe700 1 -- 172.21.15.92:0/4025634858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf300054e0 con 0x7fcf3c06d220 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf237fe700 1 -- 172.21.15.92:0/4025634858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf30003680 con 0x7fcf3c06d220 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c153e30 con 0x7fcf3c06d220 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf237fe700 1 -- 172.21.15.92:0/4025634858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30003a30 con 0x7fcf3c06d220 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.569+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c153500 con 0x7fcf3c06d220 2023-06-09T04:35:20.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.570+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c0740e0 con 0x7fcf3c06d220 2023-06-09T04:35:20.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.571+0000 7fcf237fe700 1 -- 172.21.15.92:0/4025634858 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf30003b90 con 0x7fcf3c06d220 2023-06-09T04:35:20.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.571+0000 7fcf237fe700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf2806ace0 0x7fcf2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.571+0000 7fcf237fe700 1 -- 172.21.15.92:0/4025634858 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf30088890 con 0x7fcf3c06d220 2023-06-09T04:35:20.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.571+0000 7fcf3a59c700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf2806ace0 0x7fcf2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.571+0000 7fcf3a59c700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf2806ace0 0x7fcf2806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009870 tx=0x7fcf2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:20.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.576+0000 7fcf237fe700 1 -- 172.21.15.92:0/4025634858 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcf30053940 con 0x7fcf3c06d220 2023-06-09T04:35:20.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.804+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf3c0b4fc0 con 0x7fcf3c06d220 2023-06-09T04:35:20.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.804+0000 7fcf237fe700 1 -- 172.21.15.92:0/4025634858 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7fcf30017090 con 0x7fcf3c06d220 2023-06-09T04:35:20.806 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:20.806 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:35:20.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf2806ace0 msgr2=0x7fcf2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf2806ace0 0x7fcf2806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009870 tx=0x7fcf2c008030 comp rx=0 tx=0).stop 2023-06-09T04:35:20.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c06d220 msgr2=0x7fcf3c15bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c06d220 0x7fcf3c15bb20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c4a0 tx=0x7fcf3000c7b0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 shutdown_connections 2023-06-09T04:35:20.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf2806ace0 0x7fcf2806d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf3c1638e0 0x7fcf3c163d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf3c06d220 0x7fcf3c15bb20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 --2- 172.21.15.92:0/4025634858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf3c158920 0x7fcf3c14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 >> 172.21.15.92:0/4025634858 conn(0x7fcf3c0b1800 msgr2=0x7fcf3c06aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:20.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 shutdown_connections 2023-06-09T04:35:20.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.807+0000 7fcf4118b700 1 -- 172.21.15.92:0/4025634858 wait complete. 2023-06-09T04:35:20.810 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:35:20.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:20.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.939+0000 7f9467acf700 1 Processor -- start 2023-06-09T04:35:20.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.939+0000 7f9467acf700 1 -- start start 2023-06-09T04:35:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f9467acf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94600b1360 0x7f94600b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f9467acf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94600ae630 0x7f94600aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f9467acf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600adde0 con 0x7f94600ac400 2023-06-09T04:35:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f9467acf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600ae270 con 0x7f94600b1360 2023-06-09T04:35:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f9467acf700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600af0f0 con 0x7f94600ae630 2023-06-09T04:35:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f946586b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94600b1360 0x7f94600b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f946586b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94600b1360 0x7f94600b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37536/0 (socket says 172.21.15.92:37536) 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.940+0000 7f946586b700 1 -- 172.21.15.92:0/908563461 learned_addr learned my addr 172.21.15.92:0/908563461 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f946506a700 1 --2- 172.21.15.92:0/908563461 >> v2:172.21.15.123:3300/0 conn(0x7f94600ae630 0x7f94600aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f946586b700 1 -- 172.21.15.92:0/908563461 >> v2:172.21.15.123:3300/0 conn(0x7f94600ae630 msgr2=0x7f94600aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f946586b700 1 --2- 172.21.15.92:0/908563461 >> v2:172.21.15.123:3300/0 conn(0x7f94600ae630 0x7f94600aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f946586b700 1 -- 172.21.15.92:0/908563461 >> v1:172.21.15.123:6789/0 conn(0x7f94600ac400 legacy=0x7f94600ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f946586b700 1 -- 172.21.15.92:0/908563461 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600b1c70 con 0x7f94600b1360 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f946586b700 1 --2- 172.21.15.92:0/908563461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600b1360 0x7f94600b1730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94600721a0 tx=0x7f9450009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab8d5b72ecbbf24 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:20.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9464869700 1 -- 172.21.15.92:0/908563461 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450005e30 con 0x7f94600b1360 2023-06-09T04:35:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9464869700 1 -- 172.21.15.92:0/908563461 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9450006030 con 0x7f94600b1360 2023-06-09T04:35:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9464869700 1 -- 172.21.15.92:0/908563461 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500063e0 con 0x7f94600b1360 2023-06-09T04:35:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9467acf700 1 -- 172.21.15.92:0/908563461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600b1360 msgr2=0x7f94600b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9467acf700 1 --2- 172.21.15.92:0/908563461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600b1360 0x7f94600b1730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94600721a0 tx=0x7f9450009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9467acf700 1 -- 172.21.15.92:0/908563461 shutdown_connections 2023-06-09T04:35:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9467acf700 1 --2- 172.21.15.92:0/908563461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600b1360 0x7f94600b1730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9467acf700 1 --2- 172.21.15.92:0/908563461 >> v2:172.21.15.123:3300/0 conn(0x7f94600ae630 0x7f94600aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.941+0000 7f9467acf700 1 -- 172.21.15.92:0/908563461 >> 172.21.15.92:0/908563461 conn(0x7f9460098c10 msgr2=0x7f94600cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:20.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.942+0000 7f9467acf700 1 -- 172.21.15.92:0/908563461 shutdown_connections 2023-06-09T04:35:20.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.942+0000 7f9467acf700 1 -- 172.21.15.92:0/908563461 wait complete. 2023-06-09T04:35:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 Processor -- start 2023-06-09T04:35:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 -- start start 2023-06-09T04:35:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94600ac400 0x7f94601e3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600ae630 0x7f9460156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600b1360 0x7f94600cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94601f0c90 con 0x7f94600ae630 2023-06-09T04:35:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94601efc80 con 0x7f94600ac400 2023-06-09T04:35:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f9467acf700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94601efed0 con 0x7f94600b1360 2023-06-09T04:35:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.943+0000 7f946506a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600ae630 0x7f9460156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600ae630 0x7f9460156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37542/0 (socket says 172.21.15.92:37542) 2023-06-09T04:35:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 -- 172.21.15.92:0/2421258359 learned_addr learned my addr 172.21.15.92:0/2421258359 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946606c700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600b1360 0x7f94600cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 -- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600b1360 msgr2=0x7f94600cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946586b700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94600ac400 0x7f94601e3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600b1360 0x7f94600cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 -- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94600ac400 msgr2=0x7f94601e3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94600ac400 0x7f94601e3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 -- 172.21.15.92:0/2421258359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450009500 con 0x7f94600ae630 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f946506a700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600ae630 0x7f9460156e80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c7f0 tx=0x7f945c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f94567fc700 1 -- 172.21.15.92:0/2421258359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c011800 con 0x7f94600ae630 2023-06-09T04:35:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f94567fc700 1 -- 172.21.15.92:0/2421258359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c011960 con 0x7f94600ae630 2023-06-09T04:35:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94601f01b0 con 0x7f94600ae630 2023-06-09T04:35:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f94567fc700 1 -- 172.21.15.92:0/2421258359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c011cf0 con 0x7f94600ae630 2023-06-09T04:35:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.944+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94601f0770 con 0x7f94600ae630 2023-06-09T04:35:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.945+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94601f0990 con 0x7f94600ae630 2023-06-09T04:35:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.946+0000 7f94567fc700 1 -- 172.21.15.92:0/2421258359 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f945c018360 con 0x7f94600ae630 2023-06-09T04:35:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.946+0000 7f94567fc700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f944406ace0 0x7f944406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.946+0000 7f94567fc700 1 -- 172.21.15.92:0/2421258359 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f945c089480 con 0x7f94600ae630 2023-06-09T04:35:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.946+0000 7f946586b700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f944406ace0 0x7f944406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.946+0000 7f946586b700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f944406ace0 0x7f944406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f94600721a0 tx=0x7f9450008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:20.950+0000 7f94567fc700 1 -- 172.21.15.92:0/2421258359 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f945c057fd0 con 0x7f94600ae630 2023-06-09T04:35:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.142+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94600740e0 con 0x7f94600ae630 2023-06-09T04:35:21.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.143+0000 7f94567fc700 1 -- 172.21.15.92:0/2421258359 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f945c057b60 con 0x7f94600ae630 2023-06-09T04:35:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:21.144 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:21.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.145+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f944406ace0 msgr2=0x7f944406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.145+0000 7f9467acf700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f944406ace0 0x7f944406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f94600721a0 tx=0x7f9450008070 comp rx=0 tx=0).stop 2023-06-09T04:35:21.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.145+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600ae630 msgr2=0x7f9460156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600ae630 0x7f9460156e80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f945c00c7f0 tx=0x7f945c00cbb0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 shutdown_connections 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f944406ace0 0x7f944406d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94600b1360 0x7f94600cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94600ac400 0x7f94601e3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 --2- 172.21.15.92:0/2421258359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94600ae630 0x7f9460156e80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 >> 172.21.15.92:0/2421258359 conn(0x7f9460098c10 msgr2=0x7f94600b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 shutdown_connections 2023-06-09T04:35:21.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.146+0000 7f9467acf700 1 -- 172.21.15.92:0/2421258359 wait complete. 2023-06-09T04:35:21.158 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-09T04:35:21.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T04:35:21.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.277+0000 7f67e2348700 1 Processor -- start 2023-06-09T04:35:21.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.277+0000 7f67e2348700 1 -- start start 2023-06-09T04:35:21.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67e2348700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67dc0ac400 0x7f67dc0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67e2348700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67dc0ce730 con 0x7f67dc15c7d0 2023-06-09T04:35:21.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67e2348700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67dc06aaf0 con 0x7f67dc0cf010 2023-06-09T04:35:21.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67e2348700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc15cba0 con 0x7f67dc0ac400 2023-06-09T04:35:21.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67dbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f67dc15c7d0 0x7f67dc0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35880/0 (socket says 172.21.15.92:35880) 2023-06-09T04:35:21.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67dbfff700 1 -- 172.21.15.92:0/2346284607 learned_addr learned my addr 172.21.15.92:0/2346284607 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:21.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67e08e5700 1 --2- 172.21.15.92:0/2346284607 >> v2:172.21.15.123:3300/0 conn(0x7f67dc0ac400 0x7f67dc0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67daffd700 1 -- 172.21.15.92:0/2346284607 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609626087 0 0) 0x7f67d0001290 con 0x7f67dc15c7d0 2023-06-09T04:35:21.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67daffd700 1 -- 172.21.15.92:0/2346284607 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c40035b0 con 0x7f67dc15c7d0 2023-06-09T04:35:21.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67daffd700 1 -- 172.21.15.92:0/2346284607 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140955352 0 0) 0x7f67c40035b0 con 0x7f67dc15c7d0 2023-06-09T04:35:21.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67e08e5700 1 -- 172.21.15.92:0/2346284607 >> v1:172.21.15.123:6789/0 conn(0x7f67dc0cf010 legacy=0x7f67dc06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.278+0000 7f67e08e5700 1 -- 172.21.15.92:0/2346284607 >> v1:172.21.15.92:6789/0 conn(0x7f67dc15c7d0 legacy=0x7f67dc0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e08e5700 1 -- 172.21.15.92:0/2346284607 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc069ee0 con 0x7f67dc0ac400 2023-06-09T04:35:21.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e08e5700 1 --2- 172.21.15.92:0/2346284607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc0ce1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009d90 tx=0x7f67d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=462385c2f3b8c1e9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67daffd700 1 -- 172.21.15.92:0/2346284607 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400e070 con 0x7f67dc0ac400 2023-06-09T04:35:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67daffd700 1 -- 172.21.15.92:0/2346284607 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d4003a20 con 0x7f67dc0ac400 2023-06-09T04:35:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67daffd700 1 -- 172.21.15.92:0/2346284607 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0001980 con 0x7f67dc0ac400 2023-06-09T04:35:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e2348700 1 -- 172.21.15.92:0/2346284607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 msgr2=0x7f67dc0ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e2348700 1 --2- 172.21.15.92:0/2346284607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc0ce1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009d90 tx=0x7f67d400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e2348700 1 -- 172.21.15.92:0/2346284607 shutdown_connections 2023-06-09T04:35:21.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e2348700 1 --2- 172.21.15.92:0/2346284607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc0ce1f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e2348700 1 -- 172.21.15.92:0/2346284607 >> 172.21.15.92:0/2346284607 conn(0x7f67dc098c10 msgr2=0x7f67dc0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e2348700 1 -- 172.21.15.92:0/2346284607 shutdown_connections 2023-06-09T04:35:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.279+0000 7f67e2348700 1 -- 172.21.15.92:0/2346284607 wait complete. 2023-06-09T04:35:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 Processor -- start 2023-06-09T04:35:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 -- start start 2023-06-09T04:35:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc1f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67dc0cf010 0x7f67dc15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67dc15c7d0 0x7f67dc0c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0ccc00 con 0x7f67dc0cf010 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc1f6bd0 con 0x7f67dc0ac400 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.281+0000 7f67e2348700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0c86e0 con 0x7f67dc15c7d0 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67dc0cf010 0x7f67dc15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67dc0cf010 0x7f67dc15ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37556/0 (socket says 172.21.15.92:37556) 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc1f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 -- 172.21.15.92:0/4128046125 learned_addr learned my addr 172.21.15.92:0/4128046125 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67e08e5700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67dc15c7d0 0x7f67dc0c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 -- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67dc15c7d0 msgr2=0x7f67dc0c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67dc15c7d0 0x7f67dc0c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 -- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 msgr2=0x7f67dc1f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc1f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 -- 172.21.15.92:0/4128046125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4009a70 con 0x7f67dc0cf010 2023-06-09T04:35:21.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67dbfff700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc1f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67db7fe700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67dc0cf010 0x7f67dc15ad00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00c4b0 tx=0x7f67cc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67d8ff9700 1 -- 172.21.15.92:0/4128046125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0052d0 con 0x7f67dc0cf010 2023-06-09T04:35:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67d8ff9700 1 -- 172.21.15.92:0/4128046125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc005430 con 0x7f67dc0cf010 2023-06-09T04:35:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc0c8980 con 0x7f67dc0cf010 2023-06-09T04:35:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67d8ff9700 1 -- 172.21.15.92:0/4128046125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0038c0 con 0x7f67dc0cf010 2023-06-09T04:35:21.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.282+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc0c8f40 con 0x7f67dc0cf010 2023-06-09T04:35:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.283+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc0740e0 con 0x7f67dc0cf010 2023-06-09T04:35:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.284+0000 7f67d8ff9700 1 -- 172.21.15.92:0/4128046125 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f67cc003a20 con 0x7f67dc0cf010 2023-06-09T04:35:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.284+0000 7f67d8ff9700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f67c806ad10 0x7f67c806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.284+0000 7f67d8ff9700 1 -- 172.21.15.92:0/4128046125 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67cc088780 con 0x7f67dc0cf010 2023-06-09T04:35:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.284+0000 7f67dbfff700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f67c806ad10 0x7f67c806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.284+0000 7f67dbfff700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f67c806ad10 0x7f67c806d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000ef0 tx=0x7f67d00013d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.288+0000 7f67d8ff9700 1 -- 172.21.15.92:0/4128046125 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f67cc0538c0 con 0x7f67dc0cf010 2023-06-09T04:35:21.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.472+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 --> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67dc06a4a0 con 0x7f67c806ad10 2023-06-09T04:35:21.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.480+0000 7f67d8ff9700 1 -- 172.21.15.92:0/4128046125 <== mgr.4104 v2:172.21.15.92:6832/3112104013 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271901 (secure 0 0 0) 0x7f67d000fa20 con 0x7f67c806ad10 2023-06-09T04:35:21.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f67c806ad10 msgr2=0x7f67c806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f67c806ad10 0x7f67c806d1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000ef0 tx=0x7f67d00013d0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67dc0cf010 msgr2=0x7f67dc15ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67dc0cf010 0x7f67dc15ad00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00c4b0 tx=0x7f67cc00c7c0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 shutdown_connections 2023-06-09T04:35:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f67c806ad10 0x7f67c806d1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67dc15c7d0 0x7f67dc0c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67dc0ac400 0x7f67dc1f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 --2- 172.21.15.92:0/4128046125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67dc0cf010 0x7f67dc15ad00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 >> 172.21.15.92:0/4128046125 conn(0x7f67dc098c10 msgr2=0x7f67dc0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 shutdown_connections 2023-06-09T04:35:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.483+0000 7f67e2348700 1 -- 172.21.15.92:0/4128046125 wait complete. 2023-06-09T04:35:21.486 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-06-09T04:35:21.495 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-06-09T04:35:19.844093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218372,"kb_used_data":2756,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756348,"statfs":{"total":773094113280,"available":772870500352,"internally_reserved":0,"allocated":2822144,"data_stored":1721659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001017"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:12.995653+0000","last_change":"2023-06-09T04:35:12.072943+0000","last_active":"2023-06-09T04:35:12.995653+0000","last_peered":"2023-06-09T04:35:12.995653+0000","last_clean":"2023-06-09T04:35:12.995653+0000","last_became_active":"2023-06-09T04:35:12.072623+0000","last_became_peered":"2023-06-09T04:35:12.072623+0000","last_unstale":"2023-06-09T04:35:12.995653+0000","last_undegraded":"2023-06-09T04:35:12.995653+0000","last_fullsized":"2023-06-09T04:35:12.995653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:04:02.933288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:12.996371+0000","last_change":"2023-06-09T04:35:12.002754+0000","last_active":"2023-06-09T04:35:12.996371+0000","last_peered":"2023-06-09T04:35:12.996371+0000","last_clean":"2023-06-09T04:35:12.996371+0000","last_became_active":"2023-06-09T04:35:12.002471+0000","last_became_peered":"2023-06-09T04:35:12.002471+0000","last_unstale":"2023-06-09T04:35:12.996371+0000","last_undegraded":"2023-06-09T04:35:12.996371+0000","last_fullsized":"2023-06-09T04:35:12.996371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:01:39.530762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:12.995440+0000","last_change":"2023-06-09T04:35:12.071356+0000","last_active":"2023-06-09T04:35:12.995440+0000","last_peered":"2023-06-09T04:35:12.995440+0000","last_clean":"2023-06-09T04:35:12.995440+0000","last_became_active":"2023-06-09T04:35:12.070997+0000","last_became_peered":"2023-06-09T04:35:12.070997+0000","last_unstale":"2023-06-09T04:35:12.995440+0000","last_undegraded":"2023-06-09T04:35:12.995440+0000","last_fullsized":"2023-06-09T04:35:12.995440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:01:39.530762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:12.996371+0000","last_change":"2023-06-09T04:35:11.999027+0000","last_active":"2023-06-09T04:35:12.996371+0000","last_peered":"2023-06-09T04:35:12.996371+0000","last_clean":"2023-06-09T04:35:12.996371+0000","last_became_active":"2023-06-09T04:35:11.998723+0000","last_became_peered":"2023-06-09T04:35:11.998723+0000","last_unstale":"2023-06-09T04:35:12.996371+0000","last_undegraded":"2023-06-09T04:35:12.996371+0000","last_fullsized":"2023-06-09T04:35:12.996371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:26:29.822478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":167,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:13.155529+0000","last_change":"2023-06-09T04:35:06.996716+0000","last_active":"2023-06-09T04:35:13.155529+0000","last_peered":"2023-06-09T04:35:13.155529+0000","last_clean":"2023-06-09T04:35:13.155529+0000","last_became_active":"2023-06-09T04:35:06.995565+0000","last_became_peered":"2023-06-09T04:35:06.995565+0000","last_unstale":"2023-06-09T04:35:13.155529+0000","last_undegraded":"2023-06-09T04:35:13.155529+0000","last_fullsized":"2023-06-09T04:35:13.155529+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_clean_scrub_stamp":"2023-06-09T04:35:05.906836+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:42:09.057937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:13.155596+0000","last_change":"2023-06-09T04:35:12.002654+0000","last_active":"2023-06-09T04:35:13.155596+0000","last_peered":"2023-06-09T04:35:13.155596+0000","last_clean":"2023-06-09T04:35:13.155596+0000","last_became_active":"2023-06-09T04:35:12.002320+0000","last_became_peered":"2023-06-09T04:35:12.002320+0000","last_unstale":"2023-06-09T04:35:13.155596+0000","last_undegraded":"2023-06-09T04:35:13.155596+0000","last_fullsized":"2023-06-09T04:35:13.155596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:46:15.370846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:13.155590+0000","last_change":"2023-06-09T04:35:12.071967+0000","last_active":"2023-06-09T04:35:13.155590+0000","last_peered":"2023-06-09T04:35:13.155590+0000","last_clean":"2023-06-09T04:35:13.155590+0000","last_became_active":"2023-06-09T04:35:12.071565+0000","last_became_peered":"2023-06-09T04:35:12.071565+0000","last_unstale":"2023-06-09T04:35:13.155590+0000","last_undegraded":"2023-06-09T04:35:13.155590+0000","last_fullsized":"2023-06-09T04:35:13.155590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:32:12.546172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:12.995677+0000","last_change":"2023-06-09T04:35:12.003438+0000","last_active":"2023-06-09T04:35:12.995677+0000","last_peered":"2023-06-09T04:35:12.995677+0000","last_clean":"2023-06-09T04:35:12.995677+0000","last_became_active":"2023-06-09T04:35:12.003155+0000","last_became_peered":"2023-06-09T04:35:12.003155+0000","last_unstale":"2023-06-09T04:35:12.995677+0000","last_undegraded":"2023-06-09T04:35:12.995677+0000","last_fullsized":"2023-06-09T04:35:12.995677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:26:29.822478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T04:35:13.155561+0000","last_change":"2023-06-09T04:35:12.003198+0000","last_active":"2023-06-09T04:35:13.155561+0000","last_peered":"2023-06-09T04:35:13.155561+0000","last_clean":"2023-06-09T04:35:13.155561+0000","last_became_active":"2023-06-09T04:35:12.002905+0000","last_became_peered":"2023-06-09T04:35:12.002905+0000","last_unstale":"2023-06-09T04:35:13.155561+0000","last_undegraded":"2023-06-09T04:35:13.155561+0000","last_fullsized":"2023-06-09T04:35:13.155561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:37.554276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:16.000567+0000","last_change":"2023-06-09T04:35:16.000567+0000","last_active":"2023-06-09T04:35:16.000567+0000","last_peered":"2023-06-09T04:35:16.000567+0000","last_clean":"2023-06-09T04:35:16.000567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:16.000567+0000","last_undegraded":"2023-06-09T04:35:16.000567+0000","last_fullsized":"2023-06-09T04:35:16.000567+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-09T04:35:17.006709+0000","last_change":"2023-06-09T04:35:17.006709+0000","last_active":"2023-06-09T04:35:17.006709+0000","last_peered":"2023-06-09T04:35:17.006709+0000","last_clean":"2023-06-09T04:35:17.006709+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-09T04:35:17.006709+0000","last_undegraded":"2023-06-09T04:35:17.006709+0000","last_fullsized":"2023-06-09T04:35:17.006709+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":658067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":67699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":67699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":794624,"data_stored":657398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":67699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":67699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":67699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":67699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T04:35:21.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T04:35:21.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.654+0000 7f30c9d46700 1 Processor -- start 2023-06-09T04:35:21.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.654+0000 7f30c9d46700 1 -- start start 2023-06-09T04:35:21.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.654+0000 7f30c9d46700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f30c40b1360 0x7f30c40b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.654+0000 7f30c9d46700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40aeaa0 0x7f30c40acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.654+0000 7f30c9d46700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c40adde0 con 0x7f30c40ae630 2023-06-09T04:35:21.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.654+0000 7f30c9d46700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c40ae270 con 0x7f30c40b1360 2023-06-09T04:35:21.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.654+0000 7f30c9d46700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40af0f0 con 0x7f30c40aeaa0 2023-06-09T04:35:21.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40aeaa0 0x7f30c40acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c37fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f30c40b1360 0x7f30c40b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40aeaa0 0x7f30c40acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46476/0 (socket says 172.21.15.92:46476) 2023-06-09T04:35:21.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 -- 172.21.15.92:0/255040460 learned_addr learned my addr 172.21.15.92:0/255040460 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:21.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 -- 172.21.15.92:0/255040460 >> v2:172.21.15.123:3300/0 conn(0x7f30c40b1360 msgr2=0x7f30c40b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:21.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 --2- 172.21.15.92:0/255040460 >> v2:172.21.15.123:3300/0 conn(0x7f30c40b1360 0x7f30c40b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 -- 172.21.15.92:0/255040460 >> v1:172.21.15.123:6789/0 conn(0x7f30c40ae630 legacy=0x7f30c40ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c27fc700 1 -- 172.21.15.92:0/255040460 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303465777 0 0) 0x7f30c40adde0 con 0x7f30c40ae630 2023-06-09T04:35:21.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 -- 172.21.15.92:0/255040460 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c40b1c70 con 0x7f30c40aeaa0 2023-06-09T04:35:21.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c37fe700 1 --2- 172.21.15.92:0/255040460 >> v2:172.21.15.123:3300/0 conn(0x7f30c40b1360 0x7f30c40b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:21.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.655+0000 7f30c3fff700 1 --2- 172.21.15.92:0/255040460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40aeaa0 0x7f30c40acb10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009e50 tx=0x7f30bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a54d82c0725303e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:21.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c27fc700 1 -- 172.21.15.92:0/255040460 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00e070 con 0x7f30c40aeaa0 2023-06-09T04:35:21.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c27fc700 1 -- 172.21.15.92:0/255040460 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc00b910 con 0x7f30c40aeaa0 2023-06-09T04:35:21.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c27fc700 1 -- 172.21.15.92:0/255040460 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00bcc0 con 0x7f30c40aeaa0 2023-06-09T04:35:21.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 -- 172.21.15.92:0/255040460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40aeaa0 msgr2=0x7f30c40acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 --2- 172.21.15.92:0/255040460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40aeaa0 0x7f30c40acb10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009e50 tx=0x7f30bc00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:21.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 -- 172.21.15.92:0/255040460 shutdown_connections 2023-06-09T04:35:21.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 --2- 172.21.15.92:0/255040460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40aeaa0 0x7f30c40acb10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 --2- 172.21.15.92:0/255040460 >> v2:172.21.15.123:3300/0 conn(0x7f30c40b1360 0x7f30c40b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 -- 172.21.15.92:0/255040460 >> 172.21.15.92:0/255040460 conn(0x7f30c4098c10 msgr2=0x7f30c40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:21.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 -- 172.21.15.92:0/255040460 shutdown_connections 2023-06-09T04:35:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.656+0000 7f30c9d46700 1 -- 172.21.15.92:0/255040460 wait complete. 2023-06-09T04:35:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.657+0000 7f30c9d46700 1 Processor -- start 2023-06-09T04:35:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.657+0000 7f30c9d46700 1 -- start start 2023-06-09T04:35:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c9d46700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c40ae630 0x7f30c40cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c9d46700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c40aeaa0 0x7f30c415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c9d46700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40b1360 0x7f30c41f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c9d46700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40c9250 con 0x7f30c40aeaa0 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c9d46700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41f3cf0 con 0x7f30c40ae630 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c9d46700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c41f3fd0 con 0x7f30c40b1360 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c40aeaa0 0x7f30c415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c40aeaa0 0x7f30c415a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37570/0 (socket says 172.21.15.92:37570) 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.658+0000 7f30c2ffd700 1 -- 172.21.15.92:0/2422791692 learned_addr learned my addr 172.21.15.92:0/2422791692 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c3fff700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40b1360 0x7f30c41f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c37fe700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c40ae630 0x7f30c40cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c2ffd700 1 -- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40b1360 msgr2=0x7f30c41f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c2ffd700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40b1360 0x7f30c41f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c2ffd700 1 -- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c40ae630 msgr2=0x7f30c40cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c2ffd700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c40ae630 0x7f30c40cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c2ffd700 1 -- 172.21.15.92:0/2422791692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b4009430 con 0x7f30c40aeaa0 2023-06-09T04:35:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c2ffd700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c40aeaa0 0x7f30c415a300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f30b8001eb0 tx=0x7f30b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30abfff700 1 -- 172.21.15.92:0/2422791692 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80054e0 con 0x7f30c40aeaa0 2023-06-09T04:35:21.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30abfff700 1 -- 172.21.15.92:0/2422791692 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8003680 con 0x7f30c40aeaa0 2023-06-09T04:35:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc009b30 con 0x7f30c40aeaa0 2023-06-09T04:35:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30abfff700 1 -- 172.21.15.92:0/2422791692 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b80039f0 con 0x7f30c40aeaa0 2023-06-09T04:35:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.659+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c40c3700 con 0x7f30c40aeaa0 2023-06-09T04:35:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.660+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c40740e0 con 0x7f30c40aeaa0 2023-06-09T04:35:21.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.660+0000 7f30abfff700 1 -- 172.21.15.92:0/2422791692 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30b8003b50 con 0x7f30c40aeaa0 2023-06-09T04:35:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.661+0000 7f30abfff700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f30b006ace0 0x7f30b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.661+0000 7f30abfff700 1 -- 172.21.15.92:0/2422791692 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f30b80887c0 con 0x7f30c40aeaa0 2023-06-09T04:35:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.661+0000 7f30c37fe700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f30b006ace0 0x7f30b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.661+0000 7f30c37fe700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f30b006ace0 0x7f30b006d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30c4158440 tx=0x7f30b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:21.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.665+0000 7f30abfff700 1 -- 172.21.15.92:0/2422791692 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30b80539b0 con 0x7f30c40aeaa0 2023-06-09T04:35:21.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.847+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 --> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30c40ad790 con 0x7f30b006ace0 2023-06-09T04:35:21.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.855+0000 7f30abfff700 1 -- 172.21.15.92:0/2422791692 <== mgr.4104 v2:172.21.15.92:6832/3112104013 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286723 (secure 0 0 0) 0x7f30b4011830 con 0x7f30b006ace0 2023-06-09T04:35:21.858 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:21.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f30b006ace0 msgr2=0x7f30b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f30b006ace0 0x7f30b006d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f30c4158440 tx=0x7f30b4008040 comp rx=0 tx=0).stop 2023-06-09T04:35:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c40aeaa0 msgr2=0x7f30c415a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c40aeaa0 0x7f30c415a300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f30b8001eb0 tx=0x7f30b800c8b0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 shutdown_connections 2023-06-09T04:35:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f30b006ace0 0x7f30b006d190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f30c40b1360 0x7f30c41f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30c40ae630 0x7f30c40cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 --2- 172.21.15.92:0/2422791692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30c40aeaa0 0x7f30c415a300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 >> 172.21.15.92:0/2422791692 conn(0x7f30c4098c10 msgr2=0x7f30c40b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 shutdown_connections 2023-06-09T04:35:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:21.858+0000 7f30c9d46700 1 -- 172.21.15.92:0/2422791692 wait complete. 2023-06-09T04:35:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-06-09T04:35:21.870 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-06-09T04:35:21.844466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218760,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755960,"statfs":{"total":773094113280,"available":772870103040,"internally_reserved":0,"allocated":3219456,"data_stored":1905686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.835211"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.956059+0000","last_change":"2023-06-09T04:35:17.427755+0000","last_active":"2023-06-09T04:35:18.956059+0000","last_peered":"2023-06-09T04:35:18.956059+0000","last_clean":"2023-06-09T04:35:18.956059+0000","last_became_active":"2023-06-09T04:35:17.427498+0000","last_became_peered":"2023-06-09T04:35:17.427498+0000","last_unstale":"2023-06-09T04:35:18.956059+0000","last_undegraded":"2023-06-09T04:35:18.956059+0000","last_fullsized":"2023-06-09T04:35:18.956059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:05.638304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423895+0000","last_change":"2023-06-09T04:35:17.438248+0000","last_active":"2023-06-09T04:35:18.423895+0000","last_peered":"2023-06-09T04:35:18.423895+0000","last_clean":"2023-06-09T04:35:18.423895+0000","last_became_active":"2023-06-09T04:35:17.437959+0000","last_became_peered":"2023-06-09T04:35:17.437959+0000","last_unstale":"2023-06-09T04:35:18.423895+0000","last_undegraded":"2023-06-09T04:35:18.423895+0000","last_fullsized":"2023-06-09T04:35:18.423895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071287+0000","last_change":"2023-06-09T04:35:17.444189+0000","last_active":"2023-06-09T04:35:19.071287+0000","last_peered":"2023-06-09T04:35:19.071287+0000","last_clean":"2023-06-09T04:35:19.071287+0000","last_became_active":"2023-06-09T04:35:17.443878+0000","last_became_peered":"2023-06-09T04:35:17.443878+0000","last_unstale":"2023-06-09T04:35:19.071287+0000","last_undegraded":"2023-06-09T04:35:19.071287+0000","last_fullsized":"2023-06-09T04:35:19.071287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:01.857396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423293+0000","last_change":"2023-06-09T04:35:17.439643+0000","last_active":"2023-06-09T04:35:18.423293+0000","last_peered":"2023-06-09T04:35:18.423293+0000","last_clean":"2023-06-09T04:35:18.423293+0000","last_became_active":"2023-06-09T04:35:17.439270+0000","last_became_peered":"2023-06-09T04:35:17.439270+0000","last_unstale":"2023-06-09T04:35:18.423293+0000","last_undegraded":"2023-06-09T04:35:18.423293+0000","last_fullsized":"2023-06-09T04:35:18.423293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:05.638304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.441188+0000","last_change":"2023-06-09T04:35:17.440592+0000","last_active":"2023-06-09T04:35:17.441188+0000","last_peered":"2023-06-09T04:35:17.441188+0000","last_clean":"2023-06-09T04:35:17.441188+0000","last_became_active":"2023-06-09T04:35:17.440212+0000","last_became_peered":"2023-06-09T04:35:17.440212+0000","last_unstale":"2023-06-09T04:35:17.441188+0000","last_undegraded":"2023-06-09T04:35:17.441188+0000","last_fullsized":"2023-06-09T04:35:17.441188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:20:31.980792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070853+0000","last_change":"2023-06-09T04:35:17.439583+0000","last_active":"2023-06-09T04:35:19.070853+0000","last_peered":"2023-06-09T04:35:19.070853+0000","last_clean":"2023-06-09T04:35:19.070853+0000","last_became_active":"2023-06-09T04:35:17.439241+0000","last_became_peered":"2023-06-09T04:35:17.439241+0000","last_unstale":"2023-06-09T04:35:19.070853+0000","last_undegraded":"2023-06-09T04:35:19.070853+0000","last_fullsized":"2023-06-09T04:35:19.070853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.442523+0000","last_change":"2023-06-09T04:35:17.437095+0000","last_active":"2023-06-09T04:35:17.442523+0000","last_peered":"2023-06-09T04:35:17.442523+0000","last_clean":"2023-06-09T04:35:17.442523+0000","last_became_active":"2023-06-09T04:35:17.436222+0000","last_became_peered":"2023-06-09T04:35:17.436222+0000","last_unstale":"2023-06-09T04:35:17.442523+0000","last_undegraded":"2023-06-09T04:35:17.442523+0000","last_fullsized":"2023-06-09T04:35:17.442523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:38.212529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423746+0000","last_change":"2023-06-09T04:35:17.439627+0000","last_active":"2023-06-09T04:35:18.423746+0000","last_peered":"2023-06-09T04:35:18.423746+0000","last_clean":"2023-06-09T04:35:18.423746+0000","last_became_active":"2023-06-09T04:35:17.439339+0000","last_became_peered":"2023-06-09T04:35:17.439339+0000","last_unstale":"2023-06-09T04:35:18.423746+0000","last_undegraded":"2023-06-09T04:35:18.423746+0000","last_fullsized":"2023-06-09T04:35:18.423746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.439807+0000","last_change":"2023-06-09T04:35:17.434172+0000","last_active":"2023-06-09T04:35:17.439807+0000","last_peered":"2023-06-09T04:35:17.439807+0000","last_clean":"2023-06-09T04:35:17.439807+0000","last_became_active":"2023-06-09T04:35:17.433941+0000","last_became_peered":"2023-06-09T04:35:17.433941+0000","last_unstale":"2023-06-09T04:35:17.439807+0000","last_undegraded":"2023-06-09T04:35:17.439807+0000","last_fullsized":"2023-06-09T04:35:17.439807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423982+0000","last_change":"2023-06-09T04:35:17.441671+0000","last_active":"2023-06-09T04:35:18.423982+0000","last_peered":"2023-06-09T04:35:18.423982+0000","last_clean":"2023-06-09T04:35:18.423982+0000","last_became_active":"2023-06-09T04:35:17.441387+0000","last_became_peered":"2023-06-09T04:35:17.441387+0000","last_unstale":"2023-06-09T04:35:18.423982+0000","last_undegraded":"2023-06-09T04:35:18.423982+0000","last_fullsized":"2023-06-09T04:35:18.423982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071335+0000","last_change":"2023-06-09T04:35:17.438983+0000","last_active":"2023-06-09T04:35:19.071335+0000","last_peered":"2023-06-09T04:35:19.071335+0000","last_clean":"2023-06-09T04:35:19.071335+0000","last_became_active":"2023-06-09T04:35:17.438684+0000","last_became_peered":"2023-06-09T04:35:17.438684+0000","last_unstale":"2023-06-09T04:35:19.071335+0000","last_undegraded":"2023-06-09T04:35:19.071335+0000","last_fullsized":"2023-06-09T04:35:19.071335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:15.755801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955691+0000","last_change":"2023-06-09T04:35:17.433301+0000","last_active":"2023-06-09T04:35:18.955691+0000","last_peered":"2023-06-09T04:35:18.955691+0000","last_clean":"2023-06-09T04:35:18.955691+0000","last_became_active":"2023-06-09T04:35:17.433031+0000","last_became_peered":"2023-06-09T04:35:17.433031+0000","last_unstale":"2023-06-09T04:35:18.955691+0000","last_undegraded":"2023-06-09T04:35:18.955691+0000","last_fullsized":"2023-06-09T04:35:18.955691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955577+0000","last_change":"2023-06-09T04:35:17.432672+0000","last_active":"2023-06-09T04:35:18.955577+0000","last_peered":"2023-06-09T04:35:18.955577+0000","last_clean":"2023-06-09T04:35:18.955577+0000","last_became_active":"2023-06-09T04:35:17.432388+0000","last_became_peered":"2023-06-09T04:35:17.432388+0000","last_unstale":"2023-06-09T04:35:18.955577+0000","last_undegraded":"2023-06-09T04:35:18.955577+0000","last_fullsized":"2023-06-09T04:35:18.955577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424293+0000","last_change":"2023-06-09T04:35:17.439222+0000","last_active":"2023-06-09T04:35:18.424293+0000","last_peered":"2023-06-09T04:35:18.424293+0000","last_clean":"2023-06-09T04:35:18.424293+0000","last_became_active":"2023-06-09T04:35:17.438873+0000","last_became_peered":"2023-06-09T04:35:17.438873+0000","last_unstale":"2023-06-09T04:35:18.424293+0000","last_undegraded":"2023-06-09T04:35:18.424293+0000","last_fullsized":"2023-06-09T04:35:18.424293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.442544+0000","last_change":"2023-06-09T04:35:17.437077+0000","last_active":"2023-06-09T04:35:17.442544+0000","last_peered":"2023-06-09T04:35:17.442544+0000","last_clean":"2023-06-09T04:35:17.442544+0000","last_became_active":"2023-06-09T04:35:17.436197+0000","last_became_peered":"2023-06-09T04:35:17.436197+0000","last_unstale":"2023-06-09T04:35:17.442544+0000","last_undegraded":"2023-06-09T04:35:17.442544+0000","last_fullsized":"2023-06-09T04:35:17.442544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:35:03.891489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070817+0000","last_change":"2023-06-09T04:35:17.443763+0000","last_active":"2023-06-09T04:35:19.070817+0000","last_peered":"2023-06-09T04:35:19.070817+0000","last_clean":"2023-06-09T04:35:19.070817+0000","last_became_active":"2023-06-09T04:35:17.442440+0000","last_became_peered":"2023-06-09T04:35:17.442440+0000","last_unstale":"2023-06-09T04:35:19.070817+0000","last_undegraded":"2023-06-09T04:35:19.070817+0000","last_fullsized":"2023-06-09T04:35:19.070817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:55.606991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.439789+0000","last_change":"2023-06-09T04:35:17.433713+0000","last_active":"2023-06-09T04:35:17.439789+0000","last_peered":"2023-06-09T04:35:17.439789+0000","last_clean":"2023-06-09T04:35:17.439789+0000","last_became_active":"2023-06-09T04:35:17.433428+0000","last_became_peered":"2023-06-09T04:35:17.433428+0000","last_unstale":"2023-06-09T04:35:17.439789+0000","last_undegraded":"2023-06-09T04:35:17.439789+0000","last_fullsized":"2023-06-09T04:35:17.439789+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:58.412410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071439+0000","last_change":"2023-06-09T04:35:17.432392+0000","last_active":"2023-06-09T04:35:19.071439+0000","last_peered":"2023-06-09T04:35:19.071439+0000","last_clean":"2023-06-09T04:35:19.071439+0000","last_became_active":"2023-06-09T04:35:17.432126+0000","last_became_peered":"2023-06-09T04:35:17.432126+0000","last_unstale":"2023-06-09T04:35:19.071439+0000","last_undegraded":"2023-06-09T04:35:19.071439+0000","last_fullsized":"2023-06-09T04:35:19.071439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:58:27.739525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955661+0000","last_change":"2023-06-09T04:35:17.431541+0000","last_active":"2023-06-09T04:35:18.955661+0000","last_peered":"2023-06-09T04:35:18.955661+0000","last_clean":"2023-06-09T04:35:18.955661+0000","last_became_active":"2023-06-09T04:35:17.431276+0000","last_became_peered":"2023-06-09T04:35:17.431276+0000","last_unstale":"2023-06-09T04:35:18.955661+0000","last_undegraded":"2023-06-09T04:35:18.955661+0000","last_fullsized":"2023-06-09T04:35:18.955661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:02:00.587762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071083+0000","last_change":"2023-06-09T04:35:17.445238+0000","last_active":"2023-06-09T04:35:19.071083+0000","last_peered":"2023-06-09T04:35:19.071083+0000","last_clean":"2023-06-09T04:35:19.071083+0000","last_became_active":"2023-06-09T04:35:17.444919+0000","last_became_peered":"2023-06-09T04:35:17.444919+0000","last_unstale":"2023-06-09T04:35:19.071083+0000","last_undegraded":"2023-06-09T04:35:19.071083+0000","last_fullsized":"2023-06-09T04:35:19.071083+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:04.391938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.445852+0000","last_change":"2023-06-09T04:35:17.441967+0000","last_active":"2023-06-09T04:35:17.445852+0000","last_peered":"2023-06-09T04:35:17.445852+0000","last_clean":"2023-06-09T04:35:17.445852+0000","last_became_active":"2023-06-09T04:35:17.441745+0000","last_became_peered":"2023-06-09T04:35:17.441745+0000","last_unstale":"2023-06-09T04:35:17.445852+0000","last_undegraded":"2023-06-09T04:35:17.445852+0000","last_fullsized":"2023-06-09T04:35:17.445852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:55.606991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.441158+0000","last_change":"2023-06-09T04:35:17.440445+0000","last_active":"2023-06-09T04:35:17.441158+0000","last_peered":"2023-06-09T04:35:17.441158+0000","last_clean":"2023-06-09T04:35:17.441158+0000","last_became_active":"2023-06-09T04:35:17.440125+0000","last_became_peered":"2023-06-09T04:35:17.440125+0000","last_unstale":"2023-06-09T04:35:17.441158+0000","last_undegraded":"2023-06-09T04:35:17.441158+0000","last_fullsized":"2023-06-09T04:35:17.441158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423869+0000","last_change":"2023-06-09T04:35:17.442303+0000","last_active":"2023-06-09T04:35:18.423869+0000","last_peered":"2023-06-09T04:35:18.423869+0000","last_clean":"2023-06-09T04:35:18.423869+0000","last_became_active":"2023-06-09T04:35:17.442041+0000","last_became_peered":"2023-06-09T04:35:17.442041+0000","last_unstale":"2023-06-09T04:35:18.423869+0000","last_undegraded":"2023-06-09T04:35:18.423869+0000","last_fullsized":"2023-06-09T04:35:18.423869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.442437+0000","last_change":"2023-06-09T04:35:17.436865+0000","last_active":"2023-06-09T04:35:17.442437+0000","last_peered":"2023-06-09T04:35:17.442437+0000","last_clean":"2023-06-09T04:35:17.442437+0000","last_became_active":"2023-06-09T04:35:17.436545+0000","last_became_peered":"2023-06-09T04:35:17.436545+0000","last_unstale":"2023-06-09T04:35:17.442437+0000","last_undegraded":"2023-06-09T04:35:17.442437+0000","last_fullsized":"2023-06-09T04:35:17.442437+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:43:45.048336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.445206+0000","last_change":"2023-06-09T04:35:17.442515+0000","last_active":"2023-06-09T04:35:17.445206+0000","last_peered":"2023-06-09T04:35:17.445206+0000","last_clean":"2023-06-09T04:35:17.445206+0000","last_became_active":"2023-06-09T04:35:17.442247+0000","last_became_peered":"2023-06-09T04:35:17.442247+0000","last_unstale":"2023-06-09T04:35:17.445206+0000","last_undegraded":"2023-06-09T04:35:17.445206+0000","last_fullsized":"2023-06-09T04:35:17.445206+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.443952+0000","last_change":"2023-06-09T04:35:17.435870+0000","last_active":"2023-06-09T04:35:17.443952+0000","last_peered":"2023-06-09T04:35:17.443952+0000","last_clean":"2023-06-09T04:35:17.443952+0000","last_became_active":"2023-06-09T04:35:17.435616+0000","last_became_peered":"2023-06-09T04:35:17.435616+0000","last_unstale":"2023-06-09T04:35:17.443952+0000","last_undegraded":"2023-06-09T04:35:17.443952+0000","last_fullsized":"2023-06-09T04:35:17.443952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:43.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424017+0000","last_change":"2023-06-09T04:35:17.438485+0000","last_active":"2023-06-09T04:35:18.424017+0000","last_peered":"2023-06-09T04:35:18.424017+0000","last_clean":"2023-06-09T04:35:18.424017+0000","last_became_active":"2023-06-09T04:35:17.438173+0000","last_became_peered":"2023-06-09T04:35:17.438173+0000","last_unstale":"2023-06-09T04:35:18.424017+0000","last_undegraded":"2023-06-09T04:35:18.424017+0000","last_fullsized":"2023-06-09T04:35:18.424017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:03.444952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071067+0000","last_change":"2023-06-09T04:35:17.032888+0000","last_active":"2023-06-09T04:35:19.071067+0000","last_peered":"2023-06-09T04:35:19.071067+0000","last_clean":"2023-06-09T04:35:19.071067+0000","last_became_active":"2023-06-09T04:35:17.032675+0000","last_became_peered":"2023-06-09T04:35:17.032675+0000","last_unstale":"2023-06-09T04:35:19.071067+0000","last_undegraded":"2023-06-09T04:35:19.071067+0000","last_fullsized":"2023-06-09T04:35:19.071067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:02.400588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424118+0000","last_change":"2023-06-09T04:35:17.439014+0000","last_active":"2023-06-09T04:35:18.424118+0000","last_peered":"2023-06-09T04:35:18.424118+0000","last_clean":"2023-06-09T04:35:18.424118+0000","last_became_active":"2023-06-09T04:35:17.438737+0000","last_became_peered":"2023-06-09T04:35:17.438737+0000","last_unstale":"2023-06-09T04:35:18.424118+0000","last_undegraded":"2023-06-09T04:35:18.424118+0000","last_fullsized":"2023-06-09T04:35:18.424118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:03.406729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.432604+0000","last_change":"2023-06-09T04:35:17.029890+0000","last_active":"2023-06-09T04:35:17.432604+0000","last_peered":"2023-06-09T04:35:17.432604+0000","last_clean":"2023-06-09T04:35:17.432604+0000","last_became_active":"2023-06-09T04:35:17.029430+0000","last_became_peered":"2023-06-09T04:35:17.029430+0000","last_unstale":"2023-06-09T04:35:17.432604+0000","last_undegraded":"2023-06-09T04:35:17.432604+0000","last_fullsized":"2023-06-09T04:35:17.432604+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:09.675117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.437881+0000","last_change":"2023-06-09T04:35:17.430052+0000","last_active":"2023-06-09T04:35:17.437881+0000","last_peered":"2023-06-09T04:35:17.437881+0000","last_clean":"2023-06-09T04:35:17.437881+0000","last_became_active":"2023-06-09T04:35:17.429777+0000","last_became_peered":"2023-06-09T04:35:17.429777+0000","last_unstale":"2023-06-09T04:35:17.437881+0000","last_undegraded":"2023-06-09T04:35:17.437881+0000","last_fullsized":"2023-06-09T04:35:17.437881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:25:14.183126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423796+0000","last_change":"2023-06-09T04:35:17.030206+0000","last_active":"2023-06-09T04:35:18.423796+0000","last_peered":"2023-06-09T04:35:18.423796+0000","last_clean":"2023-06-09T04:35:18.423796+0000","last_became_active":"2023-06-09T04:35:17.029820+0000","last_became_peered":"2023-06-09T04:35:17.029820+0000","last_unstale":"2023-06-09T04:35:18.423796+0000","last_undegraded":"2023-06-09T04:35:18.423796+0000","last_fullsized":"2023-06-09T04:35:18.423796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423430+0000","last_change":"2023-06-09T04:35:17.440301+0000","last_active":"2023-06-09T04:35:18.423430+0000","last_peered":"2023-06-09T04:35:18.423430+0000","last_clean":"2023-06-09T04:35:18.423430+0000","last_became_active":"2023-06-09T04:35:17.440026+0000","last_became_peered":"2023-06-09T04:35:17.440026+0000","last_unstale":"2023-06-09T04:35:18.423430+0000","last_undegraded":"2023-06-09T04:35:18.423430+0000","last_fullsized":"2023-06-09T04:35:18.423430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:32:18.360318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.420421+0000","last_change":"2023-06-09T04:35:17.029374+0000","last_active":"2023-06-09T04:35:17.420421+0000","last_peered":"2023-06-09T04:35:17.420421+0000","last_clean":"2023-06-09T04:35:17.420421+0000","last_became_active":"2023-06-09T04:35:17.029069+0000","last_became_peered":"2023-06-09T04:35:17.029069+0000","last_unstale":"2023-06-09T04:35:17.420421+0000","last_undegraded":"2023-06-09T04:35:17.420421+0000","last_fullsized":"2023-06-09T04:35:17.420421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:09.675117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423667+0000","last_change":"2023-06-09T04:35:17.436424+0000","last_active":"2023-06-09T04:35:18.423667+0000","last_peered":"2023-06-09T04:35:18.423667+0000","last_clean":"2023-06-09T04:35:18.423667+0000","last_became_active":"2023-06-09T04:35:17.436173+0000","last_became_peered":"2023-06-09T04:35:17.436173+0000","last_unstale":"2023-06-09T04:35:18.423667+0000","last_undegraded":"2023-06-09T04:35:18.423667+0000","last_fullsized":"2023-06-09T04:35:18.423667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:35:03.891489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423864+0000","last_change":"2023-06-09T04:35:17.024674+0000","last_active":"2023-06-09T04:35:18.423864+0000","last_peered":"2023-06-09T04:35:18.423864+0000","last_clean":"2023-06-09T04:35:18.423864+0000","last_became_active":"2023-06-09T04:35:17.024402+0000","last_became_peered":"2023-06-09T04:35:17.024402+0000","last_unstale":"2023-06-09T04:35:18.423864+0000","last_undegraded":"2023-06-09T04:35:18.423864+0000","last_fullsized":"2023-06-09T04:35:18.423864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423738+0000","last_change":"2023-06-09T04:35:17.431496+0000","last_active":"2023-06-09T04:35:18.423738+0000","last_peered":"2023-06-09T04:35:18.423738+0000","last_clean":"2023-06-09T04:35:18.423738+0000","last_became_active":"2023-06-09T04:35:17.431197+0000","last_became_peered":"2023-06-09T04:35:17.431197+0000","last_unstale":"2023-06-09T04:35:18.423738+0000","last_undegraded":"2023-06-09T04:35:18.423738+0000","last_fullsized":"2023-06-09T04:35:18.423738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.445697+0000","last_change":"2023-06-09T04:35:17.029451+0000","last_active":"2023-06-09T04:35:17.445697+0000","last_peered":"2023-06-09T04:35:17.445697+0000","last_clean":"2023-06-09T04:35:17.445697+0000","last_became_active":"2023-06-09T04:35:17.029169+0000","last_became_peered":"2023-06-09T04:35:17.029169+0000","last_unstale":"2023-06-09T04:35:17.445697+0000","last_undegraded":"2023-06-09T04:35:17.445697+0000","last_fullsized":"2023-06-09T04:35:17.445697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:00.832669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.444265+0000","last_change":"2023-06-09T04:35:17.438193+0000","last_active":"2023-06-09T04:35:17.444265+0000","last_peered":"2023-06-09T04:35:17.444265+0000","last_clean":"2023-06-09T04:35:17.444265+0000","last_became_active":"2023-06-09T04:35:17.437765+0000","last_became_peered":"2023-06-09T04:35:17.437765+0000","last_unstale":"2023-06-09T04:35:17.444265+0000","last_undegraded":"2023-06-09T04:35:17.444265+0000","last_fullsized":"2023-06-09T04:35:17.444265+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:20:31.980792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071305+0000","last_change":"2023-06-09T04:35:17.028442+0000","last_active":"2023-06-09T04:35:19.071305+0000","last_peered":"2023-06-09T04:35:19.071305+0000","last_clean":"2023-06-09T04:35:19.071305+0000","last_became_active":"2023-06-09T04:35:17.028162+0000","last_became_peered":"2023-06-09T04:35:17.028162+0000","last_unstale":"2023-06-09T04:35:19.071305+0000","last_undegraded":"2023-06-09T04:35:19.071305+0000","last_fullsized":"2023-06-09T04:35:19.071305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:37.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071228+0000","last_change":"2023-06-09T04:35:17.433199+0000","last_active":"2023-06-09T04:35:19.071228+0000","last_peered":"2023-06-09T04:35:19.071228+0000","last_clean":"2023-06-09T04:35:19.071228+0000","last_became_active":"2023-06-09T04:35:17.432807+0000","last_became_peered":"2023-06-09T04:35:17.432807+0000","last_unstale":"2023-06-09T04:35:19.071228+0000","last_undegraded":"2023-06-09T04:35:19.071228+0000","last_fullsized":"2023-06-09T04:35:19.071228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:56:00.781717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.441516+0000","last_change":"2023-06-09T04:35:17.029495+0000","last_active":"2023-06-09T04:35:17.441516+0000","last_peered":"2023-06-09T04:35:17.441516+0000","last_clean":"2023-06-09T04:35:17.441516+0000","last_became_active":"2023-06-09T04:35:17.028510+0000","last_became_peered":"2023-06-09T04:35:17.028510+0000","last_unstale":"2023-06-09T04:35:17.441516+0000","last_undegraded":"2023-06-09T04:35:17.441516+0000","last_fullsized":"2023-06-09T04:35:17.441516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:12.480546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423724+0000","last_change":"2023-06-09T04:35:17.436310+0000","last_active":"2023-06-09T04:35:18.423724+0000","last_peered":"2023-06-09T04:35:18.423724+0000","last_clean":"2023-06-09T04:35:18.423724+0000","last_became_active":"2023-06-09T04:35:17.436034+0000","last_became_peered":"2023-06-09T04:35:17.436034+0000","last_unstale":"2023-06-09T04:35:18.423724+0000","last_undegraded":"2023-06-09T04:35:18.423724+0000","last_fullsized":"2023-06-09T04:35:18.423724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.419538+0000","last_change":"2023-06-09T04:35:17.028971+0000","last_active":"2023-06-09T04:35:17.419538+0000","last_peered":"2023-06-09T04:35:17.419538+0000","last_clean":"2023-06-09T04:35:17.419538+0000","last_became_active":"2023-06-09T04:35:17.028680+0000","last_became_peered":"2023-06-09T04:35:17.028680+0000","last_unstale":"2023-06-09T04:35:17.419538+0000","last_undegraded":"2023-06-09T04:35:17.419538+0000","last_fullsized":"2023-06-09T04:35:17.419538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:01:59.581621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424134+0000","last_change":"2023-06-09T04:35:17.438269+0000","last_active":"2023-06-09T04:35:18.424134+0000","last_peered":"2023-06-09T04:35:18.424134+0000","last_clean":"2023-06-09T04:35:18.424134+0000","last_became_active":"2023-06-09T04:35:17.437854+0000","last_became_peered":"2023-06-09T04:35:17.437854+0000","last_unstale":"2023-06-09T04:35:18.424134+0000","last_undegraded":"2023-06-09T04:35:18.424134+0000","last_fullsized":"2023-06-09T04:35:18.424134+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.424254+0000","last_change":"2023-06-09T04:35:17.028793+0000","last_active":"2023-06-09T04:35:17.424254+0000","last_peered":"2023-06-09T04:35:17.424254+0000","last_clean":"2023-06-09T04:35:17.424254+0000","last_became_active":"2023-06-09T04:35:17.028480+0000","last_became_peered":"2023-06-09T04:35:17.028480+0000","last_unstale":"2023-06-09T04:35:17.424254+0000","last_undegraded":"2023-06-09T04:35:17.424254+0000","last_fullsized":"2023-06-09T04:35:17.424254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:38:43.929108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.437861+0000","last_change":"2023-06-09T04:35:17.427519+0000","last_active":"2023-06-09T04:35:17.437861+0000","last_peered":"2023-06-09T04:35:17.437861+0000","last_clean":"2023-06-09T04:35:17.437861+0000","last_became_active":"2023-06-09T04:35:17.427246+0000","last_became_peered":"2023-06-09T04:35:17.427246+0000","last_unstale":"2023-06-09T04:35:17.437861+0000","last_undegraded":"2023-06-09T04:35:17.437861+0000","last_fullsized":"2023-06-09T04:35:17.437861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:07:13.396488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423219+0000","last_change":"2023-06-09T04:35:17.030556+0000","last_active":"2023-06-09T04:35:18.423219+0000","last_peered":"2023-06-09T04:35:18.423219+0000","last_clean":"2023-06-09T04:35:18.423219+0000","last_became_active":"2023-06-09T04:35:17.030279+0000","last_became_peered":"2023-06-09T04:35:17.030279+0000","last_unstale":"2023-06-09T04:35:18.423219+0000","last_undegraded":"2023-06-09T04:35:18.423219+0000","last_fullsized":"2023-06-09T04:35:18.423219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:01:59.581621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424220+0000","last_change":"2023-06-09T04:35:17.431297+0000","last_active":"2023-06-09T04:35:18.424220+0000","last_peered":"2023-06-09T04:35:18.424220+0000","last_clean":"2023-06-09T04:35:18.424220+0000","last_became_active":"2023-06-09T04:35:17.431025+0000","last_became_peered":"2023-06-09T04:35:17.431025+0000","last_unstale":"2023-06-09T04:35:18.424220+0000","last_undegraded":"2023-06-09T04:35:18.424220+0000","last_fullsized":"2023-06-09T04:35:18.424220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.445393+0000","last_change":"2023-06-09T04:35:17.028833+0000","last_active":"2023-06-09T04:35:17.445393+0000","last_peered":"2023-06-09T04:35:17.445393+0000","last_clean":"2023-06-09T04:35:17.445393+0000","last_became_active":"2023-06-09T04:35:17.028375+0000","last_became_peered":"2023-06-09T04:35:17.028375+0000","last_unstale":"2023-06-09T04:35:17.445393+0000","last_undegraded":"2023-06-09T04:35:17.445393+0000","last_fullsized":"2023-06-09T04:35:17.445393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:16.412075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423784+0000","last_change":"2023-06-09T04:35:17.442725+0000","last_active":"2023-06-09T04:35:18.423784+0000","last_peered":"2023-06-09T04:35:18.423784+0000","last_clean":"2023-06-09T04:35:18.423784+0000","last_became_active":"2023-06-09T04:35:17.442455+0000","last_became_peered":"2023-06-09T04:35:17.442455+0000","last_unstale":"2023-06-09T04:35:18.423784+0000","last_undegraded":"2023-06-09T04:35:18.423784+0000","last_fullsized":"2023-06-09T04:35:18.423784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.441380+0000","last_change":"2023-06-09T04:35:17.029278+0000","last_active":"2023-06-09T04:35:17.441380+0000","last_peered":"2023-06-09T04:35:17.441380+0000","last_clean":"2023-06-09T04:35:17.441380+0000","last_became_active":"2023-06-09T04:35:17.028978+0000","last_became_peered":"2023-06-09T04:35:17.028978+0000","last_unstale":"2023-06-09T04:35:17.441380+0000","last_undegraded":"2023-06-09T04:35:17.441380+0000","last_fullsized":"2023-06-09T04:35:17.441380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:32:17.354177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070861+0000","last_change":"2023-06-09T04:35:17.438515+0000","last_active":"2023-06-09T04:35:19.070861+0000","last_peered":"2023-06-09T04:35:19.070861+0000","last_clean":"2023-06-09T04:35:19.070861+0000","last_became_active":"2023-06-09T04:35:17.437901+0000","last_became_peered":"2023-06-09T04:35:17.437901+0000","last_unstale":"2023-06-09T04:35:19.070861+0000","last_undegraded":"2023-06-09T04:35:19.070861+0000","last_fullsized":"2023-06-09T04:35:19.070861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.443777+0000","last_change":"2023-06-09T04:35:17.028112+0000","last_active":"2023-06-09T04:35:17.443777+0000","last_peered":"2023-06-09T04:35:17.443777+0000","last_clean":"2023-06-09T04:35:17.443777+0000","last_became_active":"2023-06-09T04:35:17.027836+0000","last_became_peered":"2023-06-09T04:35:17.027836+0000","last_unstale":"2023-06-09T04:35:17.443777+0000","last_undegraded":"2023-06-09T04:35:17.443777+0000","last_fullsized":"2023-06-09T04:35:17.443777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:17.978923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423291+0000","last_change":"2023-06-09T04:35:17.436148+0000","last_active":"2023-06-09T04:35:18.423291+0000","last_peered":"2023-06-09T04:35:18.423291+0000","last_clean":"2023-06-09T04:35:18.423291+0000","last_became_active":"2023-06-09T04:35:17.435824+0000","last_became_peered":"2023-06-09T04:35:17.435824+0000","last_unstale":"2023-06-09T04:35:18.423291+0000","last_undegraded":"2023-06-09T04:35:18.423291+0000","last_fullsized":"2023-06-09T04:35:18.423291+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:10.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.425087+0000","last_change":"2023-06-09T04:35:17.026454+0000","last_active":"2023-06-09T04:35:18.425087+0000","last_peered":"2023-06-09T04:35:18.425087+0000","last_clean":"2023-06-09T04:35:18.425087+0000","last_became_active":"2023-06-09T04:35:17.025128+0000","last_became_peered":"2023-06-09T04:35:17.025128+0000","last_unstale":"2023-06-09T04:35:18.425087+0000","last_undegraded":"2023-06-09T04:35:18.425087+0000","last_fullsized":"2023-06-09T04:35:18.425087+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:37.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.439771+0000","last_change":"2023-06-09T04:35:17.436924+0000","last_active":"2023-06-09T04:35:17.439771+0000","last_peered":"2023-06-09T04:35:17.439771+0000","last_clean":"2023-06-09T04:35:17.439771+0000","last_became_active":"2023-06-09T04:35:17.436657+0000","last_became_peered":"2023-06-09T04:35:17.436657+0000","last_unstale":"2023-06-09T04:35:17.439771+0000","last_undegraded":"2023-06-09T04:35:17.439771+0000","last_fullsized":"2023-06-09T04:35:17.439771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:17:39.578799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423986+0000","last_change":"2023-06-09T04:35:17.031120+0000","last_active":"2023-06-09T04:35:18.423986+0000","last_peered":"2023-06-09T04:35:18.423986+0000","last_clean":"2023-06-09T04:35:18.423986+0000","last_became_active":"2023-06-09T04:35:17.030888+0000","last_became_peered":"2023-06-09T04:35:17.030888+0000","last_unstale":"2023-06-09T04:35:18.423986+0000","last_undegraded":"2023-06-09T04:35:18.423986+0000","last_fullsized":"2023-06-09T04:35:18.423986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:05.224171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071228+0000","last_change":"2023-06-09T04:35:17.432702+0000","last_active":"2023-06-09T04:35:19.071228+0000","last_peered":"2023-06-09T04:35:19.071228+0000","last_clean":"2023-06-09T04:35:19.071228+0000","last_became_active":"2023-06-09T04:35:17.432343+0000","last_became_peered":"2023-06-09T04:35:17.432343+0000","last_unstale":"2023-06-09T04:35:19.071228+0000","last_undegraded":"2023-06-09T04:35:19.071228+0000","last_fullsized":"2023-06-09T04:35:19.071228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:25.978819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423112+0000","last_change":"2023-06-09T04:35:17.028382+0000","last_active":"2023-06-09T04:35:18.423112+0000","last_peered":"2023-06-09T04:35:18.423112+0000","last_clean":"2023-06-09T04:35:18.423112+0000","last_became_active":"2023-06-09T04:35:17.028097+0000","last_became_peered":"2023-06-09T04:35:17.028097+0000","last_unstale":"2023-06-09T04:35:18.423112+0000","last_undegraded":"2023-06-09T04:35:18.423112+0000","last_fullsized":"2023-06-09T04:35:18.423112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:57.406268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071469+0000","last_change":"2023-06-09T04:35:17.434946+0000","last_active":"2023-06-09T04:35:19.071469+0000","last_peered":"2023-06-09T04:35:19.071469+0000","last_clean":"2023-06-09T04:35:19.071469+0000","last_became_active":"2023-06-09T04:35:17.434618+0000","last_became_peered":"2023-06-09T04:35:17.434618+0000","last_unstale":"2023-06-09T04:35:19.071469+0000","last_undegraded":"2023-06-09T04:35:19.071469+0000","last_fullsized":"2023-06-09T04:35:19.071469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:07:19.431879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071066+0000","last_change":"2023-06-09T04:35:17.023550+0000","last_active":"2023-06-09T04:35:19.071066+0000","last_peered":"2023-06-09T04:35:19.071066+0000","last_clean":"2023-06-09T04:35:19.071066+0000","last_became_active":"2023-06-09T04:35:17.022566+0000","last_became_peered":"2023-06-09T04:35:17.022566+0000","last_unstale":"2023-06-09T04:35:19.071066+0000","last_undegraded":"2023-06-09T04:35:19.071066+0000","last_fullsized":"2023-06-09T04:35:19.071066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071502+0000","last_change":"2023-06-09T04:35:17.431463+0000","last_active":"2023-06-09T04:35:19.071502+0000","last_peered":"2023-06-09T04:35:19.071502+0000","last_clean":"2023-06-09T04:35:19.071502+0000","last_became_active":"2023-06-09T04:35:17.431065+0000","last_became_peered":"2023-06-09T04:35:17.431065+0000","last_unstale":"2023-06-09T04:35:19.071502+0000","last_undegraded":"2023-06-09T04:35:19.071502+0000","last_fullsized":"2023-06-09T04:35:19.071502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:53:57.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423829+0000","last_change":"2023-06-09T04:35:17.032982+0000","last_active":"2023-06-09T04:35:18.423829+0000","last_peered":"2023-06-09T04:35:18.423829+0000","last_clean":"2023-06-09T04:35:18.423829+0000","last_became_active":"2023-06-09T04:35:17.032679+0000","last_became_peered":"2023-06-09T04:35:17.032679+0000","last_unstale":"2023-06-09T04:35:18.423829+0000","last_undegraded":"2023-06-09T04:35:18.423829+0000","last_fullsized":"2023-06-09T04:35:18.423829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:54.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.445188+0000","last_change":"2023-06-09T04:35:17.442076+0000","last_active":"2023-06-09T04:35:17.445188+0000","last_peered":"2023-06-09T04:35:17.445188+0000","last_clean":"2023-06-09T04:35:17.445188+0000","last_became_active":"2023-06-09T04:35:17.441806+0000","last_became_peered":"2023-06-09T04:35:17.441806+0000","last_unstale":"2023-06-09T04:35:17.445188+0000","last_undegraded":"2023-06-09T04:35:17.445188+0000","last_fullsized":"2023-06-09T04:35:17.445188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:05:42.888858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.432595+0000","last_change":"2023-06-09T04:35:12.072943+0000","last_active":"2023-06-09T04:35:17.432595+0000","last_peered":"2023-06-09T04:35:17.432595+0000","last_clean":"2023-06-09T04:35:17.432595+0000","last_became_active":"2023-06-09T04:35:12.072623+0000","last_became_peered":"2023-06-09T04:35:12.072623+0000","last_unstale":"2023-06-09T04:35:17.432595+0000","last_undegraded":"2023-06-09T04:35:17.432595+0000","last_fullsized":"2023-06-09T04:35:17.432595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:27:49.237891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424301+0000","last_change":"2023-06-09T04:35:17.026144+0000","last_active":"2023-06-09T04:35:18.424301+0000","last_peered":"2023-06-09T04:35:18.424301+0000","last_clean":"2023-06-09T04:35:18.424301+0000","last_became_active":"2023-06-09T04:35:17.025699+0000","last_became_peered":"2023-06-09T04:35:17.025699+0000","last_unstale":"2023-06-09T04:35:18.424301+0000","last_undegraded":"2023-06-09T04:35:18.424301+0000","last_fullsized":"2023-06-09T04:35:18.424301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.445095+0000","last_change":"2023-06-09T04:35:17.440088+0000","last_active":"2023-06-09T04:35:17.445095+0000","last_peered":"2023-06-09T04:35:17.445095+0000","last_clean":"2023-06-09T04:35:17.445095+0000","last_became_active":"2023-06-09T04:35:17.439771+0000","last_became_peered":"2023-06-09T04:35:17.439771+0000","last_unstale":"2023-06-09T04:35:17.445095+0000","last_undegraded":"2023-06-09T04:35:17.445095+0000","last_fullsized":"2023-06-09T04:35:17.445095+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:33:29.054087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424339+0000","last_change":"2023-06-09T04:35:12.002754+0000","last_active":"2023-06-09T04:35:18.424339+0000","last_peered":"2023-06-09T04:35:18.424339+0000","last_clean":"2023-06-09T04:35:18.424339+0000","last_became_active":"2023-06-09T04:35:12.002471+0000","last_became_peered":"2023-06-09T04:35:12.002471+0000","last_unstale":"2023-06-09T04:35:18.424339+0000","last_undegraded":"2023-06-09T04:35:18.424339+0000","last_fullsized":"2023-06-09T04:35:18.424339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:54.754333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071063+0000","last_change":"2023-06-09T04:35:17.032034+0000","last_active":"2023-06-09T04:35:19.071063+0000","last_peered":"2023-06-09T04:35:19.071063+0000","last_clean":"2023-06-09T04:35:19.071063+0000","last_became_active":"2023-06-09T04:35:17.031771+0000","last_became_peered":"2023-06-09T04:35:17.031771+0000","last_unstale":"2023-06-09T04:35:19.071063+0000","last_undegraded":"2023-06-09T04:35:19.071063+0000","last_fullsized":"2023-06-09T04:35:19.071063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:53.206518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.444957+0000","last_change":"2023-06-09T04:35:17.439779+0000","last_active":"2023-06-09T04:35:17.444957+0000","last_peered":"2023-06-09T04:35:17.444957+0000","last_clean":"2023-06-09T04:35:17.444957+0000","last_became_active":"2023-06-09T04:35:17.439539+0000","last_became_peered":"2023-06-09T04:35:17.439539+0000","last_unstale":"2023-06-09T04:35:17.444957+0000","last_undegraded":"2023-06-09T04:35:17.444957+0000","last_fullsized":"2023-06-09T04:35:17.444957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:58.412410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955491+0000","last_change":"2023-06-09T04:35:12.071356+0000","last_active":"2023-06-09T04:35:18.955491+0000","last_peered":"2023-06-09T04:35:18.955491+0000","last_clean":"2023-06-09T04:35:18.955491+0000","last_became_active":"2023-06-09T04:35:12.070997+0000","last_became_peered":"2023-06-09T04:35:12.070997+0000","last_unstale":"2023-06-09T04:35:18.955491+0000","last_undegraded":"2023-06-09T04:35:18.955491+0000","last_fullsized":"2023-06-09T04:35:18.955491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:52.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.432611+0000","last_change":"2023-06-09T04:35:17.027888+0000","last_active":"2023-06-09T04:35:17.432611+0000","last_peered":"2023-06-09T04:35:17.432611+0000","last_clean":"2023-06-09T04:35:17.432611+0000","last_became_active":"2023-06-09T04:35:17.027381+0000","last_became_peered":"2023-06-09T04:35:17.027381+0000","last_unstale":"2023-06-09T04:35:17.432611+0000","last_undegraded":"2023-06-09T04:35:17.432611+0000","last_fullsized":"2023-06-09T04:35:17.432611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:04.632163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.445298+0000","last_change":"2023-06-09T04:35:17.444159+0000","last_active":"2023-06-09T04:35:17.445298+0000","last_peered":"2023-06-09T04:35:17.445298+0000","last_clean":"2023-06-09T04:35:17.445298+0000","last_became_active":"2023-06-09T04:35:17.443862+0000","last_became_peered":"2023-06-09T04:35:17.443862+0000","last_unstale":"2023-06-09T04:35:17.445298+0000","last_undegraded":"2023-06-09T04:35:17.445298+0000","last_fullsized":"2023-06-09T04:35:17.445298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:02:00.587762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424131+0000","last_change":"2023-06-09T04:35:11.999027+0000","last_active":"2023-06-09T04:35:18.424131+0000","last_peered":"2023-06-09T04:35:18.424131+0000","last_clean":"2023-06-09T04:35:18.424131+0000","last_became_active":"2023-06-09T04:35:11.998723+0000","last_became_peered":"2023-06-09T04:35:11.998723+0000","last_unstale":"2023-06-09T04:35:18.424131+0000","last_undegraded":"2023-06-09T04:35:18.424131+0000","last_fullsized":"2023-06-09T04:35:18.424131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:40:57.379345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.423669+0000","last_change":"2023-06-09T04:35:06.996716+0000","last_active":"2023-06-09T04:35:17.423669+0000","last_peered":"2023-06-09T04:35:17.423669+0000","last_clean":"2023-06-09T04:35:17.423669+0000","last_became_active":"2023-06-09T04:35:06.995565+0000","last_became_peered":"2023-06-09T04:35:06.995565+0000","last_unstale":"2023-06-09T04:35:17.423669+0000","last_undegraded":"2023-06-09T04:35:17.423669+0000","last_fullsized":"2023-06-09T04:35:17.423669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_clean_scrub_stamp":"2023-06-09T04:35:05.906836+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:27:44.165403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423735+0000","last_change":"2023-06-09T04:35:17.019063+0000","last_active":"2023-06-09T04:35:18.423735+0000","last_peered":"2023-06-09T04:35:18.423735+0000","last_clean":"2023-06-09T04:35:18.423735+0000","last_became_active":"2023-06-09T04:35:17.018762+0000","last_became_peered":"2023-06-09T04:35:17.018762+0000","last_unstale":"2023-06-09T04:35:18.423735+0000","last_undegraded":"2023-06-09T04:35:18.423735+0000","last_fullsized":"2023-06-09T04:35:18.423735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:29.160371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423249+0000","last_change":"2023-06-09T04:35:17.445339+0000","last_active":"2023-06-09T04:35:18.423249+0000","last_peered":"2023-06-09T04:35:18.423249+0000","last_clean":"2023-06-09T04:35:18.423249+0000","last_became_active":"2023-06-09T04:35:17.445113+0000","last_became_peered":"2023-06-09T04:35:17.445113+0000","last_unstale":"2023-06-09T04:35:18.423249+0000","last_undegraded":"2023-06-09T04:35:18.423249+0000","last_fullsized":"2023-06-09T04:35:18.423249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.424331+0000","last_change":"2023-06-09T04:35:12.002654+0000","last_active":"2023-06-09T04:35:17.424331+0000","last_peered":"2023-06-09T04:35:17.424331+0000","last_clean":"2023-06-09T04:35:17.424331+0000","last_became_active":"2023-06-09T04:35:12.002320+0000","last_became_peered":"2023-06-09T04:35:12.002320+0000","last_unstale":"2023-06-09T04:35:17.424331+0000","last_undegraded":"2023-06-09T04:35:17.424331+0000","last_fullsized":"2023-06-09T04:35:17.424331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:24.139128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.417661+0000","last_change":"2023-06-09T04:35:17.029537+0000","last_active":"2023-06-09T04:35:17.417661+0000","last_peered":"2023-06-09T04:35:17.417661+0000","last_clean":"2023-06-09T04:35:17.417661+0000","last_became_active":"2023-06-09T04:35:17.029185+0000","last_became_peered":"2023-06-09T04:35:17.029185+0000","last_unstale":"2023-06-09T04:35:17.417661+0000","last_undegraded":"2023-06-09T04:35:17.417661+0000","last_fullsized":"2023-06-09T04:35:17.417661+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:32:16.947574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.439875+0000","last_change":"2023-06-09T04:35:17.435253+0000","last_active":"2023-06-09T04:35:17.439875+0000","last_peered":"2023-06-09T04:35:17.439875+0000","last_clean":"2023-06-09T04:35:17.439875+0000","last_became_active":"2023-06-09T04:35:17.434964+0000","last_became_peered":"2023-06-09T04:35:17.434964+0000","last_unstale":"2023-06-09T04:35:17.439875+0000","last_undegraded":"2023-06-09T04:35:17.439875+0000","last_fullsized":"2023-06-09T04:35:17.439875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:58:58.290456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.432423+0000","last_change":"2023-06-09T04:35:12.071967+0000","last_active":"2023-06-09T04:35:17.432423+0000","last_peered":"2023-06-09T04:35:17.432423+0000","last_clean":"2023-06-09T04:35:17.432423+0000","last_became_active":"2023-06-09T04:35:12.071565+0000","last_became_peered":"2023-06-09T04:35:12.071565+0000","last_unstale":"2023-06-09T04:35:17.432423+0000","last_undegraded":"2023-06-09T04:35:17.432423+0000","last_fullsized":"2023-06-09T04:35:17.432423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:52.730708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.432783+0000","last_change":"2023-06-09T04:35:17.027679+0000","last_active":"2023-06-09T04:35:17.432783+0000","last_peered":"2023-06-09T04:35:17.432783+0000","last_clean":"2023-06-09T04:35:17.432783+0000","last_became_active":"2023-06-09T04:35:17.027357+0000","last_became_peered":"2023-06-09T04:35:17.027357+0000","last_unstale":"2023-06-09T04:35:17.432783+0000","last_undegraded":"2023-06-09T04:35:17.432783+0000","last_fullsized":"2023-06-09T04:35:17.432783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:37.206387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.443324+0000","last_change":"2023-06-09T04:35:17.440902+0000","last_active":"2023-06-09T04:35:17.443324+0000","last_peered":"2023-06-09T04:35:17.443324+0000","last_clean":"2023-06-09T04:35:17.443324+0000","last_became_active":"2023-06-09T04:35:17.440622+0000","last_became_peered":"2023-06-09T04:35:17.440622+0000","last_unstale":"2023-06-09T04:35:17.443324+0000","last_undegraded":"2023-06-09T04:35:17.443324+0000","last_fullsized":"2023-06-09T04:35:17.443324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:43.367718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424216+0000","last_change":"2023-06-09T04:35:17.026812+0000","last_active":"2023-06-09T04:35:18.424216+0000","last_peered":"2023-06-09T04:35:18.424216+0000","last_clean":"2023-06-09T04:35:18.424216+0000","last_became_active":"2023-06-09T04:35:17.026394+0000","last_became_peered":"2023-06-09T04:35:17.026394+0000","last_unstale":"2023-06-09T04:35:18.424216+0000","last_undegraded":"2023-06-09T04:35:18.424216+0000","last_fullsized":"2023-06-09T04:35:18.424216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.432610+0000","last_change":"2023-06-09T04:35:12.003438+0000","last_active":"2023-06-09T04:35:17.432610+0000","last_peered":"2023-06-09T04:35:17.432610+0000","last_clean":"2023-06-09T04:35:17.432610+0000","last_became_active":"2023-06-09T04:35:12.003155+0000","last_became_peered":"2023-06-09T04:35:12.003155+0000","last_unstale":"2023-06-09T04:35:17.432610+0000","last_undegraded":"2023-06-09T04:35:17.432610+0000","last_fullsized":"2023-06-09T04:35:17.432610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:32:12.332933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423524+0000","last_change":"2023-06-09T04:35:17.443931+0000","last_active":"2023-06-09T04:35:18.423524+0000","last_peered":"2023-06-09T04:35:18.423524+0000","last_clean":"2023-06-09T04:35:18.423524+0000","last_became_active":"2023-06-09T04:35:17.443663+0000","last_became_peered":"2023-06-09T04:35:17.443663+0000","last_unstale":"2023-06-09T04:35:18.423524+0000","last_undegraded":"2023-06-09T04:35:18.423524+0000","last_fullsized":"2023-06-09T04:35:18.423524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:43:45.048336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423304+0000","last_change":"2023-06-09T04:35:12.003198+0000","last_active":"2023-06-09T04:35:18.423304+0000","last_peered":"2023-06-09T04:35:18.423304+0000","last_clean":"2023-06-09T04:35:18.423304+0000","last_became_active":"2023-06-09T04:35:12.002905+0000","last_became_peered":"2023-06-09T04:35:12.002905+0000","last_unstale":"2023-06-09T04:35:18.423304+0000","last_undegraded":"2023-06-09T04:35:18.423304+0000","last_fullsized":"2023-06-09T04:35:18.423304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:27:49.237891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.418612+0000","last_change":"2023-06-09T04:35:17.033086+0000","last_active":"2023-06-09T04:35:17.418612+0000","last_peered":"2023-06-09T04:35:17.418612+0000","last_clean":"2023-06-09T04:35:17.418612+0000","last_became_active":"2023-06-09T04:35:17.032814+0000","last_became_peered":"2023-06-09T04:35:17.032814+0000","last_unstale":"2023-06-09T04:35:17.418612+0000","last_undegraded":"2023-06-09T04:35:17.418612+0000","last_fullsized":"2023-06-09T04:35:17.418612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:33:28.047946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070930+0000","last_change":"2023-06-09T04:35:17.438683+0000","last_active":"2023-06-09T04:35:19.070930+0000","last_peered":"2023-06-09T04:35:19.070930+0000","last_clean":"2023-06-09T04:35:19.070930+0000","last_became_active":"2023-06-09T04:35:17.438261+0000","last_became_peered":"2023-06-09T04:35:17.438261+0000","last_unstale":"2023-06-09T04:35:19.070930+0000","last_undegraded":"2023-06-09T04:35:19.070930+0000","last_fullsized":"2023-06-09T04:35:19.070930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:58.758092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.433345+0000","last_change":"2023-06-09T04:35:17.031891+0000","last_active":"2023-06-09T04:35:17.433345+0000","last_peered":"2023-06-09T04:35:17.433345+0000","last_clean":"2023-06-09T04:35:17.433345+0000","last_became_active":"2023-06-09T04:35:17.031613+0000","last_became_peered":"2023-06-09T04:35:17.031613+0000","last_unstale":"2023-06-09T04:35:17.433345+0000","last_undegraded":"2023-06-09T04:35:17.433345+0000","last_fullsized":"2023-06-09T04:35:17.433345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:05:41.882717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955690+0000","last_change":"2023-06-09T04:35:17.437403+0000","last_active":"2023-06-09T04:35:18.955690+0000","last_peered":"2023-06-09T04:35:18.955690+0000","last_clean":"2023-06-09T04:35:18.955690+0000","last_became_active":"2023-06-09T04:35:17.437106+0000","last_became_peered":"2023-06-09T04:35:17.437106+0000","last_unstale":"2023-06-09T04:35:18.955690+0000","last_undegraded":"2023-06-09T04:35:18.955690+0000","last_fullsized":"2023-06-09T04:35:18.955690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:10.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424990+0000","last_change":"2023-06-09T04:35:17.025288+0000","last_active":"2023-06-09T04:35:18.424990+0000","last_peered":"2023-06-09T04:35:18.424990+0000","last_clean":"2023-06-09T04:35:18.424990+0000","last_became_active":"2023-06-09T04:35:17.024862+0000","last_became_peered":"2023-06-09T04:35:17.024862+0000","last_unstale":"2023-06-09T04:35:18.424990+0000","last_undegraded":"2023-06-09T04:35:18.424990+0000","last_fullsized":"2023-06-09T04:35:18.424990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:00.851254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070993+0000","last_change":"2023-06-09T04:35:17.434826+0000","last_active":"2023-06-09T04:35:19.070993+0000","last_peered":"2023-06-09T04:35:19.070993+0000","last_clean":"2023-06-09T04:35:19.070993+0000","last_became_active":"2023-06-09T04:35:17.433157+0000","last_became_peered":"2023-06-09T04:35:17.433157+0000","last_unstale":"2023-06-09T04:35:19.070993+0000","last_undegraded":"2023-06-09T04:35:19.070993+0000","last_fullsized":"2023-06-09T04:35:19.070993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424370+0000","last_change":"2023-06-09T04:35:17.025975+0000","last_active":"2023-06-09T04:35:18.424370+0000","last_peered":"2023-06-09T04:35:18.424370+0000","last_clean":"2023-06-09T04:35:18.424370+0000","last_became_active":"2023-06-09T04:35:17.025358+0000","last_became_peered":"2023-06-09T04:35:17.025358+0000","last_unstale":"2023-06-09T04:35:18.424370+0000","last_undegraded":"2023-06-09T04:35:18.424370+0000","last_fullsized":"2023-06-09T04:35:18.424370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:24.972678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071194+0000","last_change":"2023-06-09T04:35:17.435124+0000","last_active":"2023-06-09T04:35:19.071194+0000","last_peered":"2023-06-09T04:35:19.071194+0000","last_clean":"2023-06-09T04:35:19.071194+0000","last_became_active":"2023-06-09T04:35:17.434182+0000","last_became_peered":"2023-06-09T04:35:17.434182+0000","last_unstale":"2023-06-09T04:35:19.071194+0000","last_undegraded":"2023-06-09T04:35:19.071194+0000","last_fullsized":"2023-06-09T04:35:19.071194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424031+0000","last_change":"2023-06-09T04:35:17.025538+0000","last_active":"2023-06-09T04:35:18.424031+0000","last_peered":"2023-06-09T04:35:18.424031+0000","last_clean":"2023-06-09T04:35:18.424031+0000","last_became_active":"2023-06-09T04:35:17.025170+0000","last_became_peered":"2023-06-09T04:35:17.025170+0000","last_unstale":"2023-06-09T04:35:18.424031+0000","last_undegraded":"2023-06-09T04:35:18.424031+0000","last_fullsized":"2023-06-09T04:35:18.424031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423757+0000","last_change":"2023-06-09T04:35:17.437701+0000","last_active":"2023-06-09T04:35:18.423757+0000","last_peered":"2023-06-09T04:35:18.423757+0000","last_clean":"2023-06-09T04:35:18.423757+0000","last_became_active":"2023-06-09T04:35:17.437342+0000","last_became_peered":"2023-06-09T04:35:17.437342+0000","last_unstale":"2023-06-09T04:35:18.423757+0000","last_undegraded":"2023-06-09T04:35:18.423757+0000","last_fullsized":"2023-06-09T04:35:18.423757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.440973+0000","last_change":"2023-06-09T04:35:17.029009+0000","last_active":"2023-06-09T04:35:17.440973+0000","last_peered":"2023-06-09T04:35:17.440973+0000","last_clean":"2023-06-09T04:35:17.440973+0000","last_became_active":"2023-06-09T04:35:17.028662+0000","last_became_peered":"2023-06-09T04:35:17.028662+0000","last_unstale":"2023-06-09T04:35:17.440973+0000","last_undegraded":"2023-06-09T04:35:17.440973+0000","last_fullsized":"2023-06-09T04:35:17.440973+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:04.632163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070711+0000","last_change":"2023-06-09T04:35:17.434890+0000","last_active":"2023-06-09T04:35:19.070711+0000","last_peered":"2023-06-09T04:35:19.070711+0000","last_clean":"2023-06-09T04:35:19.070711+0000","last_became_active":"2023-06-09T04:35:17.434589+0000","last_became_peered":"2023-06-09T04:35:17.434589+0000","last_unstale":"2023-06-09T04:35:19.070711+0000","last_undegraded":"2023-06-09T04:35:19.070711+0000","last_fullsized":"2023-06-09T04:35:19.070711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423112+0000","last_change":"2023-06-09T04:35:17.431610+0000","last_active":"2023-06-09T04:35:18.423112+0000","last_peered":"2023-06-09T04:35:18.423112+0000","last_clean":"2023-06-09T04:35:18.423112+0000","last_became_active":"2023-06-09T04:35:17.431333+0000","last_became_peered":"2023-06-09T04:35:17.431333+0000","last_unstale":"2023-06-09T04:35:18.423112+0000","last_undegraded":"2023-06-09T04:35:18.423112+0000","last_fullsized":"2023-06-09T04:35:18.423112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423602+0000","last_change":"2023-06-09T04:35:17.435324+0000","last_active":"2023-06-09T04:35:18.423602+0000","last_peered":"2023-06-09T04:35:18.423602+0000","last_clean":"2023-06-09T04:35:18.423602+0000","last_became_active":"2023-06-09T04:35:17.435058+0000","last_became_peered":"2023-06-09T04:35:17.435058+0000","last_unstale":"2023-06-09T04:35:18.423602+0000","last_undegraded":"2023-06-09T04:35:18.423602+0000","last_fullsized":"2023-06-09T04:35:18.423602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:38:44.935250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070935+0000","last_change":"2023-06-09T04:35:17.434438+0000","last_active":"2023-06-09T04:35:19.070935+0000","last_peered":"2023-06-09T04:35:19.070935+0000","last_clean":"2023-06-09T04:35:19.070935+0000","last_became_active":"2023-06-09T04:35:17.432923+0000","last_became_peered":"2023-06-09T04:35:17.432923+0000","last_unstale":"2023-06-09T04:35:19.070935+0000","last_undegraded":"2023-06-09T04:35:19.070935+0000","last_fullsized":"2023-06-09T04:35:19.070935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424114+0000","last_change":"2023-06-09T04:35:17.431908+0000","last_active":"2023-06-09T04:35:18.424114+0000","last_peered":"2023-06-09T04:35:18.424114+0000","last_clean":"2023-06-09T04:35:18.424114+0000","last_became_active":"2023-06-09T04:35:17.431682+0000","last_became_peered":"2023-06-09T04:35:17.431682+0000","last_unstale":"2023-06-09T04:35:18.424114+0000","last_undegraded":"2023-06-09T04:35:18.424114+0000","last_fullsized":"2023-06-09T04:35:18.424114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-06-09T04:35:17.444114+0000","last_change":"2023-06-09T04:35:17.435983+0000","last_active":"2023-06-09T04:35:17.444114+0000","last_peered":"2023-06-09T04:35:17.444114+0000","last_clean":"2023-06-09T04:35:17.444114+0000","last_became_active":"2023-06-09T04:35:17.435723+0000","last_became_peered":"2023-06-09T04:35:17.435723+0000","last_unstale":"2023-06-09T04:35:17.444114+0000","last_undegraded":"2023-06-09T04:35:17.444114+0000","last_fullsized":"2023-06-09T04:35:17.444114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":679765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":677898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":87617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":94360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":89237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":94922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T04:35:21.878 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-09T04:35:21.878 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-09T04:35:21.879 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-06-09T04:35:21.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-06-09T04:35:21.880 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-06-09T04:35:22.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.031+0000 7f9826111700 1 Processor -- start 2023-06-09T04:35:22.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.031+0000 7f9826111700 1 -- start start 2023-06-09T04:35:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f9826111700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f982015c7d0 0x7f98200ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f9826111700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200cf010 0x7f982006e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f9826111700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98200ce730 con 0x7f98200ac400 2023-06-09T04:35:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f9826111700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982006aaf0 con 0x7f982015c7d0 2023-06-09T04:35:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f9826111700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982015cba0 con 0x7f98200cf010 2023-06-09T04:35:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f982015c7d0 0x7f98200ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f982015c7d0 0x7f98200ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37576/0 (socket says 172.21.15.92:37576) 2023-06-09T04:35:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 -- 172.21.15.92:0/2843714849 learned_addr learned my addr 172.21.15.92:0/2843714849 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981ffff700 1 --2- 172.21.15.92:0/2843714849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200cf010 0x7f982006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 -- 172.21.15.92:0/2843714849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200cf010 msgr2=0x7f982006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 --2- 172.21.15.92:0/2843714849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200cf010 0x7f982006e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 -- 172.21.15.92:0/2843714849 >> v1:172.21.15.123:6789/0 conn(0x7f98200ac400 legacy=0x7f98200ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 -- 172.21.15.92:0/2843714849 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9820069ee0 con 0x7f982015c7d0 2023-06-09T04:35:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981ffff700 1 --2- 172.21.15.92:0/2843714849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200cf010 0x7f982006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981f7fe700 1 --2- 172.21.15.92:0/2843714849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f982015c7d0 0x7f98200ce1f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9810006fd0 tx=0x7f9810009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7312da7beb9e45 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981e7fc700 1 -- 172.21.15.92:0/2843714849 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810005e30 con 0x7f982015c7d0 2023-06-09T04:35:22.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981e7fc700 1 -- 172.21.15.92:0/2843714849 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810006030 con 0x7f982015c7d0 2023-06-09T04:35:22.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.032+0000 7f981e7fc700 1 -- 172.21.15.92:0/2843714849 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98100063c0 con 0x7f982015c7d0 2023-06-09T04:35:22.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 -- 172.21.15.92:0/2843714849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f982015c7d0 msgr2=0x7f98200ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 --2- 172.21.15.92:0/2843714849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f982015c7d0 0x7f98200ce1f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9810006fd0 tx=0x7f9810009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 -- 172.21.15.92:0/2843714849 shutdown_connections 2023-06-09T04:35:22.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 --2- 172.21.15.92:0/2843714849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200cf010 0x7f982006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 --2- 172.21.15.92:0/2843714849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f982015c7d0 0x7f98200ce1f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 -- 172.21.15.92:0/2843714849 >> 172.21.15.92:0/2843714849 conn(0x7f9820098c10 msgr2=0x7f98200cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:22.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 -- 172.21.15.92:0/2843714849 shutdown_connections 2023-06-09T04:35:22.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.033+0000 7f9826111700 1 -- 172.21.15.92:0/2843714849 wait complete. 2023-06-09T04:35:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.034+0000 7f9826111700 1 Processor -- start 2023-06-09T04:35:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.034+0000 7f9826111700 1 -- start start 2023-06-09T04:35:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f9826111700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200ac400 0x7f98201f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f9826111700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98200cf010 0x7f98200b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f9826111700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982015c7d0 0x7f9820159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f9826111700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98200c4870 con 0x7f98200cf010 2023-06-09T04:35:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f9826111700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98201f5fb0 con 0x7f982015c7d0 2023-06-09T04:35:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f9826111700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98201f6290 con 0x7f98200ac400 2023-06-09T04:35:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98200cf010 0x7f98200b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98200cf010 0x7f98200b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37584/0 (socket says 172.21.15.92:37584) 2023-06-09T04:35:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 -- 172.21.15.92:0/3098022831 learned_addr learned my addr 172.21.15.92:0/3098022831 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981f7fe700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200ac400 0x7f98201f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981ffff700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982015c7d0 0x7f9820159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 -- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200ac400 msgr2=0x7f98201f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200ac400 0x7f98201f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 -- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982015c7d0 msgr2=0x7f9820159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982015c7d0 0x7f9820159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.035+0000 7f981effd700 1 -- 172.21.15.92:0/3098022831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9810009500 con 0x7f98200cf010 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.036+0000 7f981effd700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98200cf010 0x7f98200b0e00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f981400c4d0 tx=0x7f981400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.036+0000 7f9807fff700 1 -- 172.21.15.92:0/3098022831 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814011790 con 0x7f98200cf010 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.036+0000 7f9807fff700 1 -- 172.21.15.92:0/3098022831 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140118f0 con 0x7f98200cf010 2023-06-09T04:35:22.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.036+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98201f6570 con 0x7f98200cf010 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.036+0000 7f9807fff700 1 -- 172.21.15.92:0/3098022831 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814011ca0 con 0x7f98200cf010 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.036+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98201f5ca0 con 0x7f98200cf010 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.037+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98200c49b0 con 0x7f98200cf010 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.037+0000 7f9807fff700 1 -- 172.21.15.92:0/3098022831 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f981401c460 con 0x7f98200cf010 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.037+0000 7f9807fff700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f980c06ace0 0x7f980c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.038+0000 7f981f7fe700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f980c06ace0 0x7f980c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.038+0000 7f9807fff700 1 -- 172.21.15.92:0/3098022831 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f98140892a0 con 0x7f98200cf010 2023-06-09T04:35:22.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.038+0000 7f981f7fe700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f980c06ace0 0x7f980c06d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9810000c00 tx=0x7f9810008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.042+0000 7f9807fff700 1 -- 172.21.15.92:0/3098022831 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f98200c49b0 con 0x7f98200cf010 2023-06-09T04:35:22.126 INFO:tasks.ceph.mds.d.smithi123.stderr:2023-06-09T04:35:22.126+0000 7fc947681700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T04:35:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.286+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f98200c49b0 con 0x7f98200cf010 2023-06-09T04:35:22.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.289+0000 7f9807fff700 1 -- 172.21.15.92:0/3098022831 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170842 (secure 0 0 0) 0x7f98200c49b0 con 0x7f98200cf010 2023-06-09T04:35:22.291 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f980c06ace0 msgr2=0x7f980c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f980c06ace0 0x7f980c06d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9810000c00 tx=0x7f9810008070 comp rx=0 tx=0).stop 2023-06-09T04:35:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98200cf010 msgr2=0x7f98200b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98200cf010 0x7f98200b0e00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f981400c4d0 tx=0x7f981400c890 comp rx=0 tx=0).stop 2023-06-09T04:35:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 shutdown_connections 2023-06-09T04:35:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f980c06ace0 0x7f980c06d190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f982015c7d0 0x7f9820159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98200ac400 0x7f98201f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 --2- 172.21.15.92:0/3098022831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98200cf010 0x7f98200b0e00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 >> 172.21.15.92:0/3098022831 conn(0x7f9820098c10 msgr2=0x7f98200ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 shutdown_connections 2023-06-09T04:35:22.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.292+0000 7f9826111700 1 -- 172.21.15.92:0/3098022831 wait complete. 2023-06-09T04:35:22.304 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3112104013},{"type":"v1","addr":"172.21.15.92:6833","nonce":3112104013}]},"active_addr":"172.21.15.92:6833/3112104013","active_change":"2023-06-09T04:34:57.835108+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2348402201}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1692579462}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2914647587}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2507468513}]}]} 2023-06-09T04:35:22.308 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-06-09T04:35:22.308 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-06-09T04:35:22.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:22.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.427+0000 7f8fd8a9d700 1 Processor -- start 2023-06-09T04:35:22.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.427+0000 7f8fd8a9d700 1 -- start start 2023-06-09T04:35:22.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd8a9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd8a9d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd40ce730 con 0x7f8fd40ac400 2023-06-09T04:35:22.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd8a9d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd406aaf0 con 0x7f8fd40cf010 2023-06-09T04:35:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd8a9d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd415cba0 con 0x7f8fd415c7d0 2023-06-09T04:35:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd1d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8fd40ac400 0x7f8fd40ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35922/0 (socket says 172.21.15.92:35922) 2023-06-09T04:35:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd1d9b700 1 -- 172.21.15.92:0/1966303549 learned_addr learned my addr 172.21.15.92:0/1966303549 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd259c700 1 --2- 172.21.15.92:0/1966303549 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280945434 0 0) 0x7f8fc4001110 con 0x7f8fd40ac400 2023-06-09T04:35:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fbc0035b0 con 0x7f8fd40ac400 2023-06-09T04:35:22.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603592029 0 0) 0x7f8fd406aaf0 con 0x7f8fd40cf010 2023-06-09T04:35:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.428+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc4001110 con 0x7f8fd40cf010 2023-06-09T04:35:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd259c700 1 -- 172.21.15.92:0/1966303549 >> v1:172.21.15.123:6789/0 conn(0x7f8fd40cf010 legacy=0x7f8fd406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 249729732 0 0) 0x7f8fbc0035b0 con 0x7f8fd40ac400 2023-06-09T04:35:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd259c700 1 -- 172.21.15.92:0/1966303549 >> v1:172.21.15.92:6789/0 conn(0x7f8fd40ac400 legacy=0x7f8fd40ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd259c700 1 -- 172.21.15.92:0/1966303549 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd4069ee0 con 0x7f8fd415c7d0 2023-06-09T04:35:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd259c700 1 --2- 172.21.15.92:0/1966303549 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40ce1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009850 tx=0x7f8fc8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a502b7ee96eaece4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4001800 con 0x7f8fd415c7d0 2023-06-09T04:35:22.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc8005c30 con 0x7f8fd415c7d0 2023-06-09T04:35:22.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd159a700 1 -- 172.21.15.92:0/1966303549 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8004030 con 0x7f8fd415c7d0 2023-06-09T04:35:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/1966303549 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 msgr2=0x7f8fd40ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/1966303549 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40ce1f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009850 tx=0x7f8fc8009b60 comp rx=0 tx=0).stop 2023-06-09T04:35:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/1966303549 shutdown_connections 2023-06-09T04:35:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.429+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/1966303549 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40ce1f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.430+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/1966303549 >> 172.21.15.92:0/1966303549 conn(0x7f8fd4098c10 msgr2=0x7f8fd40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.430+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/1966303549 shutdown_connections 2023-06-09T04:35:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.430+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/1966303549 wait complete. 2023-06-09T04:35:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.431+0000 7f8fd8a9d700 1 Processor -- start 2023-06-09T04:35:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.431+0000 7f8fd8a9d700 1 -- start start 2023-06-09T04:35:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fd40ac400 0x7f8fd40c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd40cf010 0x7f8fd40b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40c3420 con 0x7f8fd40ac400 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40c3700 con 0x7f8fd40cf010 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40c39e0 con 0x7f8fd415c7d0 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fd40ac400 0x7f8fd40c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fd40ac400 0x7f8fd40c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37596/0 (socket says 172.21.15.92:37596) 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 -- 172.21.15.92:0/2928574477 learned_addr learned my addr 172.21.15.92:0/2928574477 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd2d9d700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd1d9b700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd40cf010 0x7f8fd40b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 -- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 msgr2=0x7f8fd40cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 -- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd40cf010 msgr2=0x7f8fd40b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd40cf010 0x7f8fd40b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 -- 172.21.15.92:0/2928574477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc8009430 con 0x7f8fd40ac400 2023-06-09T04:35:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd259c700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fd40ac400 0x7f8fd40c44c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40b24d0 tx=0x7f8fc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fc2ffd700 1 -- 172.21.15.92:0/2928574477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8003b70 con 0x7f8fd40ac400 2023-06-09T04:35:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fc2ffd700 1 -- 172.21.15.92:0/2928574477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc8003cd0 con 0x7f8fd40ac400 2023-06-09T04:35:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd40c3cc0 con 0x7f8fd40ac400 2023-06-09T04:35:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fc2ffd700 1 -- 172.21.15.92:0/2928574477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8011640 con 0x7f8fd40ac400 2023-06-09T04:35:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.432+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd41561a0 con 0x7f8fd40ac400 2023-06-09T04:35:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.433+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd4156360 con 0x7f8fd40ac400 2023-06-09T04:35:22.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.434+0000 7f8fc2ffd700 1 -- 172.21.15.92:0/2928574477 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8fc8018030 con 0x7f8fd40ac400 2023-06-09T04:35:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.434+0000 7f8fc2ffd700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8fb406ace0 0x7f8fb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.434+0000 7f8fd1d9b700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8fb406ace0 0x7f8fb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.435+0000 7f8fd1d9b700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8fb406ace0 0x7f8fb406d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40c9e00 tx=0x7f8fc4001c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.435+0000 7f8fc2ffd700 1 -- 172.21.15.92:0/2928574477 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8fc8003e40 con 0x7f8fd40ac400 2023-06-09T04:35:22.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.439+0000 7f8fc2ffd700 1 -- 172.21.15.92:0/2928574477 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8fc8055750 con 0x7f8fd40ac400 2023-06-09T04:35:22.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.627+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fd40740e0 con 0x7f8fd40ac400 2023-06-09T04:35:22.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.628+0000 7f8fc2ffd700 1 -- 172.21.15.92:0/2928574477 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f8fc802b020 con 0x7f8fd40ac400 2023-06-09T04:35:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:22.629 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8fb406ace0 msgr2=0x7f8fb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8fb406ace0 0x7f8fb406d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40c9e00 tx=0x7f8fc4001c90 comp rx=0 tx=0).stop 2023-06-09T04:35:22.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fd40ac400 msgr2=0x7f8fd40c44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fd40ac400 0x7f8fd40c44c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40b24d0 tx=0x7f8fc8004750 comp rx=0 tx=0).stop 2023-06-09T04:35:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 shutdown_connections 2023-06-09T04:35:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8fb406ace0 0x7f8fb406d190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8fd415c7d0 0x7f8fd40cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8fd40ac400 0x7f8fd40c44c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 --2- 172.21.15.92:0/2928574477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fd40cf010 0x7f8fd40b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 >> 172.21.15.92:0/2928574477 conn(0x7f8fd4098c10 msgr2=0x7f8fd40ac810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 shutdown_connections 2023-06-09T04:35:22.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.630+0000 7f8fd8a9d700 1 -- 172.21.15.92:0/2928574477 wait complete. 2023-06-09T04:35:22.642 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-06-09T04:35:22.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:22.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.761+0000 7f4aee243700 1 Processor -- start 2023-06-09T04:35:22.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.761+0000 7f4aee243700 1 -- start start 2023-06-09T04:35:22.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4aee243700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4ae815a500 0x7f4ae80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4aee243700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a8d0 0x7f4ae8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4aee243700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ae80cbf50 0x7f4ae806a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4aee243700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae80ac400 con 0x7f4ae80cbf50 2023-06-09T04:35:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4aee243700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae80cd7e0 con 0x7f4ae815a500 2023-06-09T04:35:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4aee243700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae80cdda0 con 0x7f4ae815a8d0 2023-06-09T04:35:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ae80cbf50 0x7f4ae806a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ae80cbf50 0x7f4ae806a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37608/0 (socket says 172.21.15.92:37608) 2023-06-09T04:35:22.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2712289952 learned_addr learned my addr 172.21.15.92:0/2712289952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:22.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae6ffd700 1 --2- 172.21.15.92:0/2712289952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a8d0 0x7f4ae8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae77fe700 1 --2- 172.21.15.92:0/2712289952 >> v2:172.21.15.123:3300/0 conn(0x7f4ae815a500 0x7f4ae80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2712289952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a8d0 msgr2=0x7f4ae8069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:22.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 --2- 172.21.15.92:0/2712289952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a8d0 0x7f4ae8069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2712289952 >> v2:172.21.15.123:3300/0 conn(0x7f4ae815a500 msgr2=0x7f4ae80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:22.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 --2- 172.21.15.92:0/2712289952 >> v2:172.21.15.123:3300/0 conn(0x7f4ae815a500 0x7f4ae80cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2712289952 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae80aeca0 con 0x7f4ae80cbf50 2023-06-09T04:35:22.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae7fff700 1 --2- 172.21.15.92:0/2712289952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae80cbf50 0x7f4ae806a2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80cd4d0 tx=0x7f4ae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3843e8b3049d6c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae67fc700 1 -- 172.21.15.92:0/2712289952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000bbc0 con 0x7f4ae80cbf50 2023-06-09T04:35:22.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae67fc700 1 -- 172.21.15.92:0/2712289952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae000bd20 con 0x7f4ae80cbf50 2023-06-09T04:35:22.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.762+0000 7f4ae67fc700 1 -- 172.21.15.92:0/2712289952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0010710 con 0x7f4ae80cbf50 2023-06-09T04:35:22.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 -- 172.21.15.92:0/2712289952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae80cbf50 msgr2=0x7f4ae806a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 --2- 172.21.15.92:0/2712289952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae80cbf50 0x7f4ae806a2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80cd4d0 tx=0x7f4ae000b330 comp rx=0 tx=0).stop 2023-06-09T04:35:22.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 -- 172.21.15.92:0/2712289952 shutdown_connections 2023-06-09T04:35:22.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 --2- 172.21.15.92:0/2712289952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae80cbf50 0x7f4ae806a2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 --2- 172.21.15.92:0/2712289952 >> v2:172.21.15.123:3300/0 conn(0x7f4ae815a500 0x7f4ae80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 --2- 172.21.15.92:0/2712289952 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a8d0 0x7f4ae8069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 -- 172.21.15.92:0/2712289952 >> 172.21.15.92:0/2712289952 conn(0x7f4ae8098c10 msgr2=0x7f4ae80cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:22.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 -- 172.21.15.92:0/2712289952 shutdown_connections 2023-06-09T04:35:22.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.763+0000 7f4aee243700 1 -- 172.21.15.92:0/2712289952 wait complete. 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.764+0000 7f4aee243700 1 Processor -- start 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.764+0000 7f4aee243700 1 -- start start 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4aee243700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ae80cbf50 0x7f4ae80afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4aee243700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a500 0x7f4ae81e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4aee243700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae815a8d0 0x7f4ae81e1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4aee243700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae81e1300 con 0x7f4ae815a8d0 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4aee243700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae81e2920 con 0x7f4ae80cbf50 2023-06-09T04:35:22.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4aee243700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae80b00c0 con 0x7f4ae815a500 2023-06-09T04:35:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae815a8d0 0x7f4ae81e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae815a8d0 0x7f4ae81e1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37614/0 (socket says 172.21.15.92:37614) 2023-06-09T04:35:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2554481943 learned_addr learned my addr 172.21.15.92:0/2554481943 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae6ffd700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a500 0x7f4ae81e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae77fe700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ae80cbf50 0x7f4ae80afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a500 msgr2=0x7f4ae81e23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a500 0x7f4ae81e23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ae80cbf50 msgr2=0x7f4ae80afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ae80cbf50 0x7f4ae80afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.765+0000 7f4ae7fff700 1 -- 172.21.15.92:0/2554481943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae0009b30 con 0x7f4ae815a8d0 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.766+0000 7f4ae7fff700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae815a8d0 0x7f4ae81e1790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000b370 tx=0x7f4ae0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.766+0000 7f4acffff700 1 -- 172.21.15.92:0/2554481943 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0004250 con 0x7f4ae815a8d0 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.766+0000 7f4acffff700 1 -- 172.21.15.92:0/2554481943 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae00043d0 con 0x7f4ae815a8d0 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.766+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae80b03a0 con 0x7f4ae815a8d0 2023-06-09T04:35:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.766+0000 7f4acffff700 1 -- 172.21.15.92:0/2554481943 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0004780 con 0x7f4ae815a8d0 2023-06-09T04:35:22.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.766+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae81e0f90 con 0x7f4ae815a8d0 2023-06-09T04:35:22.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.767+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae81e0b10 con 0x7f4ae815a8d0 2023-06-09T04:35:22.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.767+0000 7f4acffff700 1 -- 172.21.15.92:0/2554481943 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ae00104d0 con 0x7f4ae815a8d0 2023-06-09T04:35:22.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.767+0000 7f4acffff700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4ad406ace0 0x7f4ad406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:22.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.767+0000 7f4acffff700 1 -- 172.21.15.92:0/2554481943 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ae008f060 con 0x7f4ae815a8d0 2023-06-09T04:35:22.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.767+0000 7f4ae77fe700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4ad406ace0 0x7f4ad406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:22.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.768+0000 7f4ae77fe700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4ad406ace0 0x7f4ad406d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8006fd0 tx=0x7f4ad8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:22.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.772+0000 7f4acffff700 1 -- 172.21.15.92:0/2554481943 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ae81e0b10 con 0x7f4ae815a8d0 2023-06-09T04:35:22.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.960+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ae81e0b10 con 0x7f4ae815a8d0 2023-06-09T04:35:22.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.961+0000 7f4acffff700 1 -- 172.21.15.92:0/2554481943 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f4ae81e0b10 con 0x7f4ae815a8d0 2023-06-09T04:35:22.961 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:22.962 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:22.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4ad406ace0 msgr2=0x7f4ad406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4ad406ace0 0x7f4ad406d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8006fd0 tx=0x7f4ad8008040 comp rx=0 tx=0).stop 2023-06-09T04:35:22.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae815a8d0 msgr2=0x7f4ae81e1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:22.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae815a8d0 0x7f4ae81e1790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000b370 tx=0x7f4ae0004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 shutdown_connections 2023-06-09T04:35:22.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4ad406ace0 0x7f4ad406d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ae815a8d0 0x7f4ae81e1790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ae80cbf50 0x7f4ae80afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 --2- 172.21.15.92:0/2554481943 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ae815a500 0x7f4ae81e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:22.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 >> 172.21.15.92:0/2554481943 conn(0x7f4ae8098c10 msgr2=0x7f4ae806a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 shutdown_connections 2023-06-09T04:35:22.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:22.963+0000 7f4aee243700 1 -- 172.21.15.92:0/2554481943 wait complete. 2023-06-09T04:35:22.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-06-09T04:35:22.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-06-09T04:35:22.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-06-09T04:35:22.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-06-09T04:35:22.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-06-09T04:35:22.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-06-09T04:35:22.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-06-09T04:35:22.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-06-09T04:35:23.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.175+0000 7ff8c8681700 1 Processor -- start 2023-06-09T04:35:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.175+0000 7ff8c8681700 1 -- start start 2023-06-09T04:35:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c8681700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8c00cc9a0 0x7ff8c0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c8681700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff8c006d810 0x7ff8c006a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c8681700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c00ccd70 con 0x7ff8c016cbb0 2023-06-09T04:35:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c8681700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c0169510 con 0x7ff8c00cc9a0 2023-06-09T04:35:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c8681700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c006c320 con 0x7ff8c006d810 2023-06-09T04:35:23.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c641d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff8c00cc9a0 0x7ff8c0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c5c1c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff8c016cbb0 0x7ff8c00702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35952/0 (socket says 172.21.15.92:35952) 2023-06-09T04:35:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.177+0000 7ff8c5c1c700 1 -- 172.21.15.92:0/2680249419 learned_addr learned my addr 172.21.15.92:0/2680249419 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.178+0000 7ff8c541b700 1 -- 172.21.15.92:0/2680249419 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646640042 0 0) 0x7ff8bc001160 con 0x7ff8c016cbb0 2023-06-09T04:35:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.178+0000 7ff8c541b700 1 -- 172.21.15.92:0/2680249419 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8ac0079c0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.178+0000 7ff8c641d700 1 -- 172.21.15.92:0/2680249419 >> v2:172.21.15.123:3300/0 conn(0x7ff8c006d810 msgr2=0x7ff8c006a2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T04:35:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.178+0000 7ff8c641d700 1 --2- 172.21.15.92:0/2680249419 >> v2:172.21.15.123:3300/0 conn(0x7ff8c006d810 0x7ff8c006a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.178+0000 7ff8c641d700 1 -- 172.21.15.92:0/2680249419 >> v1:172.21.15.92:6789/0 conn(0x7ff8c016cbb0 legacy=0x7ff8c00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.178+0000 7ff8c641d700 1 -- 172.21.15.92:0/2680249419 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c016d0e0 con 0x7ff8c00cc9a0 2023-06-09T04:35:23.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.178+0000 7ff8c641d700 1 --2- 172.21.15.92:0/2680249419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c00cc9a0 0x7ff8c0069db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff8c006bbd0 tx=0x7ff8b800b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635bdec23b8d583d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.182+0000 7ff8c541b700 1 -- 172.21.15.92:0/2680249419 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800bd30 con 0x7ff8c00cc9a0 2023-06-09T04:35:23.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.182+0000 7ff8c541b700 1 -- 172.21.15.92:0/2680249419 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b8008e90 con 0x7ff8c00cc9a0 2023-06-09T04:35:23.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.182+0000 7ff8c541b700 1 -- 172.21.15.92:0/2680249419 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8011690 con 0x7ff8c00cc9a0 2023-06-09T04:35:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.183+0000 7ff8c8681700 1 -- 172.21.15.92:0/2680249419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c00cc9a0 msgr2=0x7ff8c0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.183+0000 7ff8c8681700 1 --2- 172.21.15.92:0/2680249419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c00cc9a0 0x7ff8c0069db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff8c006bbd0 tx=0x7ff8b800b340 comp rx=0 tx=0).stop 2023-06-09T04:35:23.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.184+0000 7ff8c8681700 1 -- 172.21.15.92:0/2680249419 shutdown_connections 2023-06-09T04:35:23.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.184+0000 7ff8c8681700 1 --2- 172.21.15.92:0/2680249419 >> v2:172.21.15.123:3300/0 conn(0x7ff8c006d810 0x7ff8c006a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.184+0000 7ff8c8681700 1 --2- 172.21.15.92:0/2680249419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c00cc9a0 0x7ff8c0069db0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.184+0000 7ff8c8681700 1 -- 172.21.15.92:0/2680249419 >> 172.21.15.92:0/2680249419 conn(0x7ff8c00b0ff0 msgr2=0x7ff8c00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.185+0000 7ff8c8681700 1 -- 172.21.15.92:0/2680249419 shutdown_connections 2023-06-09T04:35:23.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.185+0000 7ff8c8681700 1 -- 172.21.15.92:0/2680249419 wait complete. 2023-06-09T04:35:23.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.186+0000 7f088d28c700 1 Processor -- start 2023-06-09T04:35:23.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.186+0000 7f088d28c700 1 -- start start 2023-06-09T04:35:23.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.186+0000 7ff8c8681700 1 Processor -- start 2023-06-09T04:35:23.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.187+0000 7ff8c8681700 1 -- start start 2023-06-09T04:35:23.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c8681700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c006d810 0x7ff8c0206f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c8681700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c00cc9a0 0x7ff8c015f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c8681700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8c016cbb0 0x7ff8c00c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c8681700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00c3a40 con 0x7ff8c006d810 2023-06-09T04:35:23.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c8681700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00c3d20 con 0x7ff8c00cc9a0 2023-06-09T04:35:23.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c8681700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00c4000 con 0x7ff8c016cbb0 2023-06-09T04:35:23.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.187+0000 7f088d28c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f08880ae630 0x7f08880aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.187+0000 7f088d28c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880cd7e0 0x7f08880cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.187+0000 7f088d28c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880adde0 con 0x7f08880b1360 2023-06-09T04:35:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.187+0000 7f088d28c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880ae270 con 0x7f08880ae630 2023-06-09T04:35:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.187+0000 7f088d28c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880af0f0 con 0x7f08880cd7e0 2023-06-09T04:35:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08877fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880cd7e0 0x7f08880cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08877fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880cd7e0 0x7f08880cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46536/0 (socket says 172.21.15.92:46536) 2023-06-09T04:35:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08877fe700 1 -- 172.21.15.92:0/3215610434 learned_addr learned my addr 172.21.15.92:0/3215610434 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08867fc700 1 --2- 172.21.15.92:0/3215610434 >> v2:172.21.15.123:3300/0 conn(0x7f08880ae630 0x7f08880aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c6c1e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8c016cbb0 0x7ff8c00c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c6c1e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8c016cbb0 0x7ff8c00c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46544/0 (socket says 172.21.15.92:46544) 2023-06-09T04:35:23.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c6c1e700 1 -- 172.21.15.92:0/1014612536 learned_addr learned my addr 172.21.15.92:0/1014612536 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7ff8c641d700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c006d810 0x7ff8c0206f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08877fe700 1 -- 172.21.15.92:0/3215610434 >> v2:172.21.15.123:3300/0 conn(0x7f08880ae630 msgr2=0x7f08880aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08877fe700 1 --2- 172.21.15.92:0/3215610434 >> v2:172.21.15.123:3300/0 conn(0x7f08880ae630 0x7f08880aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08877fe700 1 -- 172.21.15.92:0/3215610434 >> v1:172.21.15.92:6789/0 conn(0x7f08880b1360 legacy=0x7f08880b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.188+0000 7f08877fe700 1 -- 172.21.15.92:0/3215610434 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880ad4b0 con 0x7f08880cd7e0 2023-06-09T04:35:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c6c1e700 1 -- 172.21.15.92:0/1014612536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c00cc9a0 msgr2=0x7ff8c015f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c6c1e700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c00cc9a0 0x7ff8c015f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c6c1e700 1 -- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c006d810 msgr2=0x7ff8c0206f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c6c1e700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8c006d810 0x7ff8c0206f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c6c1e700 1 -- 172.21.15.92:0/1014612536 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b80089d0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c6c1e700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8c016cbb0 0x7ff8c00c4ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000b3e0 tx=0x7ff8b000b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7f08877fe700 1 --2- 172.21.15.92:0/3215610434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880cd7e0 0x7f08880cdc60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f088000d150 tx=0x7f088000d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14543548779bd6f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000bea0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b0011830 con 0x7ff8c016cbb0 2023-06-09T04:35:23.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0011be0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c00c42e0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.189+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c02074d0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.190+0000 7f31ba461700 1 Processor -- start 2023-06-09T04:35:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.190+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8c016d6a0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.190+0000 7f0885ffb700 1 -- 172.21.15.92:0/3215610434 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880010070 con 0x7f08880cd7e0 2023-06-09T04:35:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.190+0000 7f0885ffb700 1 -- 172.21.15.92:0/3215610434 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088000dcd0 con 0x7f08880cd7e0 2023-06-09T04:35:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.190+0000 7f0885ffb700 1 -- 172.21.15.92:0/3215610434 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088000b310 con 0x7f08880cd7e0 2023-06-09T04:35:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 -- 172.21.15.92:0/3215610434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880cd7e0 msgr2=0x7f08880cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 --2- 172.21.15.92:0/3215610434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880cd7e0 0x7f08880cdc60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f088000d150 tx=0x7f088000d460 comp rx=0 tx=0).stop 2023-06-09T04:35:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f2ed960d700 1 Processor -- start 2023-06-09T04:35:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f2ed960d700 1 -- start start 2023-06-09T04:35:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 -- 172.21.15.92:0/3215610434 shutdown_connections 2023-06-09T04:35:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 --2- 172.21.15.92:0/3215610434 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880cd7e0 0x7f08880cdc60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 --2- 172.21.15.92:0/3215610434 >> v2:172.21.15.123:3300/0 conn(0x7f08880ae630 0x7f08880aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 -- 172.21.15.92:0/3215610434 >> 172.21.15.92:0/3215610434 conn(0x7f0888098c10 msgr2=0x7f08880cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 -- 172.21.15.92:0/3215610434 shutdown_connections 2023-06-09T04:35:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f088d28c700 1 -- 172.21.15.92:0/3215610434 wait complete. 2023-06-09T04:35:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.190+0000 7f31ba461700 1 -- start start 2023-06-09T04:35:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f31ba461700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b40c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f31ba461700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b40d06e0 con 0x7f31b406a3f0 2023-06-09T04:35:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f31ba461700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b40ce560 con 0x7f31b41643f0 2023-06-09T04:35:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f31ba461700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40c3240 con 0x7f31b406a020 2023-06-09T04:35:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f1f21ce0700 1 Processor -- start 2023-06-09T04:35:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.191+0000 7f31b3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b40c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b40c2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46546/0 (socket says 172.21.15.92:46546) 2023-06-09T04:35:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b3fff700 1 -- 172.21.15.92:0/1357428868 learned_addr learned my addr 172.21.15.92:0/1357428868 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f2ed960d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2ed416d020 0x7f2ed406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f2ed960d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed4169510 con 0x7f2ed4069db0 2023-06-09T04:35:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f2ed960d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed406c830 con 0x7f2ed416cbb0 2023-06-09T04:35:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f2ed960d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed416db50 con 0x7f2ed416d020 2023-06-09T04:35:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b2ffd700 1 -- 172.21.15.92:0/1357428868 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249189347 0 0) 0x7f31b40d06e0 con 0x7f31b406a3f0 2023-06-09T04:35:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b2ffd700 1 -- 172.21.15.92:0/1357428868 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f319c0035b0 con 0x7f31b406a3f0 2023-06-09T04:35:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b3fff700 1 -- 172.21.15.92:0/1357428868 >> v1:172.21.15.123:6789/0 conn(0x7f31b41643f0 legacy=0x7f31b406d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b3fff700 1 -- 172.21.15.92:0/1357428868 >> v1:172.21.15.92:6789/0 conn(0x7f31b406a3f0 legacy=0x7f31b4163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b3fff700 1 -- 172.21.15.92:0/1357428868 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40ce840 con 0x7f31b406a020 2023-06-09T04:35:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f2ed2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2ed4069db0 0x7f2ed406a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35974/0 (socket says 172.21.15.92:35974) 2023-06-09T04:35:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f2ed2ffd700 1 -- 172.21.15.92:0/679592791 learned_addr learned my addr 172.21.15.92:0/679592791 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f31b3fff700 1 --2- 172.21.15.92:0/1357428868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b40c2d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f31a40098f0 tx=0x7f31a4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b2263e7f11303bc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f2ed37fe700 1 --2- 172.21.15.92:0/679592791 >> v2:172.21.15.92:3300/0 conn(0x7f2ed416d020 0x7f2ed406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed1ffb700 1 -- 172.21.15.92:0/679592791 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900014019 0 0) 0x7f2ec8001110 con 0x7f2ed4069db0 2023-06-09T04:35:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed1ffb700 1 -- 172.21.15.92:0/679592791 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ebc0035b0 con 0x7f2ed4069db0 2023-06-09T04:35:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed37fe700 1 -- 172.21.15.92:0/679592791 >> v1:172.21.15.123:6789/0 conn(0x7f2ed416cbb0 legacy=0x7f2ed406ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed37fe700 1 -- 172.21.15.92:0/679592791 >> v1:172.21.15.92:6789/0 conn(0x7f2ed4069db0 legacy=0x7f2ed406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed37fe700 1 -- 172.21.15.92:0/679592791 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40cccf0 con 0x7f2ed416d020 2023-06-09T04:35:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f088d28c700 1 Processor -- start 2023-06-09T04:35:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed37fe700 1 --2- 172.21.15.92:0/679592791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed416d020 0x7f2ed406d3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d1e0 tx=0x7f2ecc00d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1228b1bf12b8727d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f31b2ffd700 1 -- 172.21.15.92:0/1357428868 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4005e30 con 0x7f31b406a020 2023-06-09T04:35:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.192+0000 7f1f21ce0700 1 -- start start 2023-06-09T04:35:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f1f21ce0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1f1c070cd0 0x7f1f1c15c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f1f21ce0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f1c15cbc0 0x7f1f1c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f1f21ce0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c06aa10 con 0x7f1f1c1586f0 2023-06-09T04:35:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f1f21ce0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1c0ae180 con 0x7f1f1c070cd0 2023-06-09T04:35:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f1f21ce0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1c06a170 con 0x7f1f1c15cbc0 2023-06-09T04:35:23.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed1ffb700 1 -- 172.21.15.92:0/679592791 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc010070 con 0x7f2ed416d020 2023-06-09T04:35:23.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed1ffb700 1 -- 172.21.15.92:0/679592791 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc00dbb0 con 0x7f2ed416d020 2023-06-09T04:35:23.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f2ed1ffb700 1 -- 172.21.15.92:0/679592791 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc007aa0 con 0x7f2ed416d020 2023-06-09T04:35:23.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f31b2ffd700 1 -- 172.21.15.92:0/1357428868 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a4006030 con 0x7f31b406a020 2023-06-09T04:35:23.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f31b2ffd700 1 -- 172.21.15.92:0/1357428868 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40063c0 con 0x7f31b406a020 2023-06-09T04:35:23.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f1f1b7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1f1c1586f0 0x7f1f1c0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58476/0 (socket says 172.21.15.92:58476) 2023-06-09T04:35:23.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f1f1b7fe700 1 -- 172.21.15.92:0/4046836382 learned_addr learned my addr 172.21.15.92:0/4046836382 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.193+0000 7f088d28c700 1 -- start start 2023-06-09T04:35:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f1b3f731700 1 Processor -- start 2023-06-09T04:35:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f31ba461700 1 -- 172.21.15.92:0/1357428868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 msgr2=0x7f31b40c2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f31ba461700 1 --2- 172.21.15.92:0/1357428868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b40c2d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f31a40098f0 tx=0x7f31a4009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f31ba461700 1 -- 172.21.15.92:0/1357428868 shutdown_connections 2023-06-09T04:35:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f31ba461700 1 --2- 172.21.15.92:0/1357428868 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b40c2d00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f31ba461700 1 -- 172.21.15.92:0/1357428868 >> 172.21.15.92:0/1357428868 conn(0x7f31b4098c10 msgr2=0x7f31b40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f2ed960d700 1 -- 172.21.15.92:0/679592791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed416d020 msgr2=0x7f2ed406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f2ed960d700 1 --2- 172.21.15.92:0/679592791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed416d020 0x7f2ed406d3d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00d1e0 tx=0x7f2ecc00d4f0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036188676 0 0) 0x7f1f1c06aa10 con 0x7f1f1c1586f0 2023-06-09T04:35:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f040035b0 con 0x7f1f1c1586f0 2023-06-09T04:35:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f2ed960d700 1 -- 172.21.15.92:0/679592791 shutdown_connections 2023-06-09T04:35:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f2ed960d700 1 --2- 172.21.15.92:0/679592791 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed416d020 0x7f2ed406d3d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f2ed960d700 1 -- 172.21.15.92:0/679592791 >> 172.21.15.92:0/679592791 conn(0x7f2ed40b0ff0 msgr2=0x7f2ed40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f2ed960d700 1 -- 172.21.15.92:0/679592791 shutdown_connections 2023-06-09T04:35:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f088d28c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880ae630 0x7f08881f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f088d28c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08880b1360 0x7f08880b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f088d28c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cd7e0 0x7f08880b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8b0011d40 con 0x7ff8c016cbb0 2023-06-09T04:35:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f088d28c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880c4100 con 0x7f08880cd7e0 2023-06-09T04:35:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f088d28c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880c43e0 con 0x7f08880b1360 2023-06-09T04:35:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f088d28c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880c46c0 con 0x7f08880ae630 2023-06-09T04:35:23.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f31ba461700 1 -- 172.21.15.92:0/1357428868 shutdown_connections 2023-06-09T04:35:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08880b1360 0x7f08880b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08880b1360 0x7f08880b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58062/0 (socket says 172.21.15.92:58062) 2023-06-09T04:35:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 -- 172.21.15.92:0/3751747085 learned_addr learned my addr 172.21.15.92:0/3751747085 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f0886ffd700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880ae630 0x7f08881f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f2ed960d700 1 -- 172.21.15.92:0/679592791 wait complete. 2023-06-09T04:35:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f31ba461700 1 -- 172.21.15.92:0/1357428868 wait complete. 2023-06-09T04:35:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 415650737 0 0) 0x7f1f040035b0 con 0x7f1f1c1586f0 2023-06-09T04:35:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 >> v2:172.21.15.123:3300/0 conn(0x7f1f1c15cbc0 msgr2=0x7f1f1c0cc590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T04:35:23.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 --2- 172.21.15.92:0/4046836382 >> v2:172.21.15.123:3300/0 conn(0x7f1f1c15cbc0 0x7f1f1c0cc590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 >> v2:172.21.15.92:3300/0 conn(0x7f1f1c070cd0 msgr2=0x7f1f1c15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 --2- 172.21.15.92:0/4046836382 >> v2:172.21.15.92:3300/0 conn(0x7f1f1c070cd0 0x7f1f1c15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c06a450 con 0x7f1f1c1586f0 2023-06-09T04:35:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f1f0c001670 con 0x7f1f1c1586f0 2023-06-09T04:35:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 >> v1:172.21.15.123:6789/0 conn(0x7f1f1c1586f0 legacy=0x7f1f1c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f040037f0 0x7f1f04003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f04004190 0x7f1f0400b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0400b690 0x7f1f0400db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c06a170 con 0x7f1f04004190 2023-06-09T04:35:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0400e470 con 0x7f1f0400b690 2023-06-09T04:35:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0400e630 con 0x7f1f040037f0 2023-06-09T04:35:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 -- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880ae630 msgr2=0x7f08881f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08880ae630 0x7f08881f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 -- 172.21.15.92:0/3751747085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cd7e0 msgr2=0x7f08880b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08880cd7e0 0x7f08880b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 -- 172.21.15.92:0/3751747085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088000bd40 con 0x7f08880b1360 2023-06-09T04:35:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7ff8b6ffd700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff8a406ace0 0x7ff8a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8b00899a0 con 0x7ff8c016cbb0 2023-06-09T04:35:23.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7ff8b6ffd700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6808/3209682425,v1:172.21.15.92:6809/3209682425] conn(0x7ff8a4073590 0x7ff8a40759a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 --> [v2:172.21.15.92:6808/3209682425,v1:172.21.15.92:6809/3209682425] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8a4076050 con 0x7ff8a4073590 2023-06-09T04:35:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b0089d50 con 0x7ff8c016cbb0 2023-06-09T04:35:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.194+0000 7f1b3f731700 1 -- start start 2023-06-09T04:35:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f1b3f731700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b381586f0 0x7f1b38070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.195+0000 7f08867fc700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08880b1360 0x7f08880b2710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f087800e9b0 tx=0x7f087800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1b3f731700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b38071210 con 0x7f1b380c92d0 2023-06-09T04:35:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1b3f731700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b3806aa10 con 0x7f1b380c8e80 2023-06-09T04:35:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1b3f731700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b380ae180 con 0x7f1b381586f0 2023-06-09T04:35:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7ff8c641d700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff8a406ace0 0x7ff8a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7ff8c5c1c700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6808/3209682425,v1:172.21.15.92:6809/3209682425] conn(0x7ff8a4073590 0x7ff8a40759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1f1b7fe700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f040037f0 0x7f1f04003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1b3dcce700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b380c92d0 0x7f1b380cc590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35998/0 (socket says 172.21.15.92:35998) 2023-06-09T04:35:23.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1067997417 learned_addr learned my addr 172.21.15.92:0/1067997417 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800c850 con 0x7f08880b1360 2023-06-09T04:35:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800c9b0 con 0x7f08880b1360 2023-06-09T04:35:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800cd20 con 0x7f08880b1360 2023-06-09T04:35:23.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7ff8c641d700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff8a406ace0 0x7ff8a406d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8008cc0 tx=0x7ff8b801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316849433 0 0) 0x7f1b28001110 con 0x7f1b380c92d0 2023-06-09T04:35:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b240035b0 con 0x7f1b380c92d0 2023-06-09T04:35:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08880c49a0 con 0x7f08880b1360 2023-06-09T04:35:23.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08880c3d90 con 0x7f08880b1360 2023-06-09T04:35:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f31ba461700 1 Processor -- start 2023-06-09T04:35:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1f1affd700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f04004190 0x7f1f0400b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.196+0000 7f2ed960d700 1 Processor -- start 2023-06-09T04:35:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f31ba461700 1 -- start start 2023-06-09T04:35:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7ff8c5c1c700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6808/3209682425,v1:172.21.15.92:6809/3209682425] conn(0x7ff8a4073590 0x7ff8a40759a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1f1b7fe700 1 -- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0400b690 msgr2=0x7f1f0400db40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:35:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1f1b7fe700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f0400b690 0x7f1f0400db40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1f1b7fe700 1 -- 172.21.15.92:0/4046836382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f04004190 msgr2=0x7f1f0400b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1f1b7fe700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f04004190 0x7f1f0400b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1f1b7fe700 1 -- 172.21.15.92:0/4046836382 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c06a450 con 0x7f1f040037f0 2023-06-09T04:35:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499155566 0 0) 0x7f1b3806aa10 con 0x7f1b380c8e80 2023-06-09T04:35:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b28001110 con 0x7f1b380c8e80 2023-06-09T04:35:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880668197 0 0) 0x7f1b240035b0 con 0x7f1b380c92d0 2023-06-09T04:35:23.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 >> v2:172.21.15.92:3300/0 conn(0x7f1b381586f0 msgr2=0x7f1b38070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 --2- 172.21.15.92:0/1067997417 >> v2:172.21.15.92:3300/0 conn(0x7f1b381586f0 0x7f1b38070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 >> v1:172.21.15.123:6789/0 conn(0x7f1b380c8e80 legacy=0x7f1b3815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380c9750 con 0x7f1b380c92d0 2023-06-09T04:35:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f1b28001780 con 0x7f1b380c92d0 2023-06-09T04:35:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 >> v1:172.21.15.92:6789/0 conn(0x7f1b380c92d0 legacy=0x7f1b380cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b24003b90 0x7f1b24003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24004850 0x7f1b24004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b24005250 0x7f1b2400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2400c150 con 0x7f1b24003b90 2023-06-09T04:35:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2400c2c0 con 0x7f1b24004850 2023-06-09T04:35:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2400c480 con 0x7f1b24005250 2023-06-09T04:35:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b3dcce700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b24005250 0x7f1b2400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b3cccc700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24004850 0x7f1b24004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f1b3d4cd700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b24003b90 0x7f1b24003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.197+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08880ada70 con 0x7f08880b1360 2023-06-09T04:35:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24004850 msgr2=0x7f1b24004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b3dcce700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b24004850 0x7f1b24004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1067997417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b24003b90 msgr2=0x7f1b24003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b3dcce700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b24003b90 0x7f1b24003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1067997417 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380c9750 con 0x7f1b24005250 2023-06-09T04:35:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1067997417 reap_dead start 2023-06-09T04:35:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== osd.0 v2:172.21.15.92:6808/3209682425 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff8a4076050 con 0x7ff8a4073590 2023-06-09T04:35:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1f1bfff700 1 -- 172.21.15.92:0/4046836382 reap_dead start 2023-06-09T04:35:23.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1f1b7fe700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f040037f0 0x7f1f04003c00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00c370 tx=0x7f1f04004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f88eb200df0f16 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b3dcce700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b24005250 0x7f1b2400ba10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b28005070 tx=0x7f1b24005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783ebb275be91381 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f2ed960d700 1 -- start start 2023-06-09T04:35:23.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31ba461700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b4157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31ba461700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31b406a3f0 0x7f31b40c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31ba461700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41643f0 0x7f31b40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31ba461700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41f63c0 con 0x7f31b406a3f0 2023-06-09T04:35:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31ba461700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41f65a0 con 0x7f31b41643f0 2023-06-09T04:35:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31ba461700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41f6880 con 0x7f31b406a020 2023-06-09T04:35:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31b3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b4157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31b3fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b4157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46590/0 (socket says 172.21.15.92:46590) 2023-06-09T04:35:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31b3fff700 1 -- 172.21.15.92:0/824531975 learned_addr learned my addr 172.21.15.92:0/824531975 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31b89fe700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41643f0 0x7f31b40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f31b37fe700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31b406a3f0 0x7f31b40c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed960d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4069db0 0x7f2ed40c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed960d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed416cbb0 0x7f2ed40b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed960d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ed416d020 0x7f2ed4207750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed960d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed40c1b90 con 0x7f2ed4069db0 2023-06-09T04:35:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed960d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4208330 con 0x7f2ed416cbb0 2023-06-09T04:35:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed960d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4208610 con 0x7f2ed416d020 2023-06-09T04:35:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ed416d020 0x7f2ed4207750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ed416d020 0x7f2ed4207750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46596/0 (socket says 172.21.15.92:46596) 2023-06-09T04:35:23.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 -- 172.21.15.92:0/2253508265 learned_addr learned my addr 172.21.15.92:0/2253508265 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c005960 con 0x7f1f040037f0 2023-06-09T04:35:23.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800de30 con 0x7f1b24005250 2023-06-09T04:35:23.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2800f400 con 0x7f1b24005250 2023-06-09T04:35:23.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.198+0000 7f1b2ffff700 1 -- 172.21.15.92:0/1067997417 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800f7b0 con 0x7f1b24005250 2023-06-09T04:35:23.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f1b3f731700 1 -- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b24005250 msgr2=0x7f1b2400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f1b3f731700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b24005250 0x7f1b2400ba10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b28005070 tx=0x7f1b24005880 comp rx=0 tx=0).stop 2023-06-09T04:35:23.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c005ac0 con 0x7f1f040037f0 2023-06-09T04:35:23.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f1f1a7fc700 1 -- 172.21.15.92:0/4046836382 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c005e70 con 0x7f1f040037f0 2023-06-09T04:35:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 -- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed416cbb0 msgr2=0x7f2ed40b3c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:35:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ed416cbb0 0x7f2ed40b3c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 -- 172.21.15.92:0/2253508265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4069db0 msgr2=0x7f2ed40c3330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:35:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2ed4069db0 0x7f2ed40c3330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.199+0000 7f2ed37fe700 1 -- 172.21.15.92:0/2253508265 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc007910 con 0x7f2ed416d020 2023-06-09T04:35:23.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1b3f731700 1 -- 172.21.15.92:0/1067997417 shutdown_connections 2023-06-09T04:35:23.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1b3f731700 1 --2- 172.21.15.92:0/1067997417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b24005250 0x7f1b2400ba10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1b3f731700 1 -- 172.21.15.92:0/1067997417 >> 172.21.15.92:0/1067997417 conn(0x7f1b380b0ff0 msgr2=0x7f1b380ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f2ed37fe700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ed416d020 0x7f2ed4207750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc018040 tx=0x7f2ecc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc004510 con 0x7f2ed416d020 2023-06-09T04:35:23.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc004670 con 0x7f2ed416d020 2023-06-09T04:35:23.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed41696b0 con 0x7f2ed416d020 2023-06-09T04:35:23.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed4169c70 con 0x7f2ed416d020 2023-06-09T04:35:23.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1f21ce0700 1 -- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f040037f0 msgr2=0x7f1f04003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1f21ce0700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f040037f0 0x7f1f04003c00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00c370 tx=0x7f1f04004870 comp rx=0 tx=0).stop 2023-06-09T04:35:23.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1f21ce0700 1 -- 172.21.15.92:0/4046836382 shutdown_connections 2023-06-09T04:35:23.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1f21ce0700 1 --2- 172.21.15.92:0/4046836382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f040037f0 0x7f1f04003c00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1f21ce0700 1 -- 172.21.15.92:0/4046836382 >> 172.21.15.92:0/4046836382 conn(0x7f1f1c0b0ff0 msgr2=0x7f1f1c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0878013470 con 0x7f08880b1360 2023-06-09T04:35:23.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc008030 con 0x7f2ed416d020 2023-06-09T04:35:23.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1b3f731700 1 -- 172.21.15.92:0/1067997417 shutdown_connections 2023-06-09T04:35:23.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.200+0000 7f1b3f731700 1 -- 172.21.15.92:0/1067997417 wait complete. 2023-06-09T04:35:23.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.201+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ed416d870 con 0x7f2ed416d020 2023-06-09T04:35:23.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f1b3f731700 1 Processor -- start 2023-06-09T04:35:23.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f31b37fe700 1 -- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 msgr2=0x7f31b4157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f31b37fe700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31b406a020 0x7f31b4157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f31b37fe700 1 -- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41643f0 msgr2=0x7f31b40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f31b37fe700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b41643f0 0x7f31b40cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f31b37fe700 1 -- 172.21.15.92:0/824531975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4009500 con 0x7f31b406a3f0 2023-06-09T04:35:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.201+0000 7f086f7fe700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f087406adb0 0x7f087406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.201+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0878089420 con 0x7f08880b1360 2023-06-09T04:35:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.201+0000 7f0886ffd700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f087406adb0 0x7f087406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.201+0000 7f086f7fe700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:6800/909955472,v1:172.21.15.123:6801/909955472] conn(0x7f0874073650 0x7f0874075a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.201+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 --> [v2:172.21.15.123:6800/909955472,v1:172.21.15.123:6801/909955472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0874076110 con 0x7f0874073650 2023-06-09T04:35:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.201+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f087808ba40 con 0x7f08880b1360 2023-06-09T04:35:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f0886ffd700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f087406adb0 0x7f087406d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f087c007fd0 tx=0x7f087c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.202+0000 7f08877fe700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:6800/909955472,v1:172.21.15.123:6801/909955472] conn(0x7f0874073650 0x7f0874075a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.203+0000 7f31b37fe700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31b406a3f0 0x7f31b40c4f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0012e0 tx=0x7f31ac00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.203+0000 7f08877fe700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:6800/909955472,v1:172.21.15.123:6801/909955472] conn(0x7f0874073650 0x7f0874075a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.203+0000 7f1f21ce0700 1 -- 172.21.15.92:0/4046836382 shutdown_connections 2023-06-09T04:35:23.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.203+0000 7f1f21ce0700 1 -- 172.21.15.92:0/4046836382 wait complete. 2023-06-09T04:35:23.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.203+0000 7f1b3f731700 1 -- start start 2023-06-09T04:35:23.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3f731700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b24005250 0x7f1b380cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3f731700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b380c62b0 0x7f1b380c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3f731700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b380c7c00 0x7f1b380c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3f731700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380cd050 con 0x7f1b24005250 2023-06-09T04:35:23.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3f731700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b381f2330 con 0x7f1b380c62b0 2023-06-09T04:35:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3f731700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b381f1570 con 0x7f1b380c7c00 2023-06-09T04:35:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.203+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ecc008190 con 0x7f2ed416d020 2023-06-09T04:35:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f2ebb7fe700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2ec006ace0 0x7f2ec006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ecc08b440 con 0x7f2ed416d020 2023-06-09T04:35:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f2ebb7fe700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:6808/3070586225,v1:172.21.15.123:6809/3070586225] conn(0x7f2ec0073510 0x7f2ec0075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 --> [v2:172.21.15.123:6808/3070586225,v1:172.21.15.123:6809/3070586225] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ec0075fd0 con 0x7f2ec0073510 2023-06-09T04:35:23.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ecc08b7f0 con 0x7f2ed416d020 2023-06-09T04:35:23.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== osd.4 v2:172.21.15.123:6800/909955472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0874076110 con 0x7f0874073650 2023-06-09T04:35:23.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3dcce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b380c7c00 0x7f1b380c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3dcce700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b380c7c00 0x7f1b380c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46606/0 (socket says 172.21.15.92:46606) 2023-06-09T04:35:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.204+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1808431036 learned_addr learned my addr 172.21.15.92:0/1808431036 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1808431036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b380c62b0 msgr2=0x7f1b380c3b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T04:35:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1b3dcce700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b380c62b0 0x7f1b380c3b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b24005250 msgr2=0x7f1b380cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1b3dcce700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b24005250 0x7f1b380cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1808431036 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b381f1850 con 0x7f1b380c7c00 2023-06-09T04:35:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1f21ce0700 1 Processor -- start 2023-06-09T04:35:23.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1b3dcce700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b380c7c00 0x7f1b380c8060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1b380c2330 tx=0x7f1b28010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f2ed27fc700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:6808/3070586225,v1:172.21.15.123:6809/3070586225] conn(0x7f2ec0073510 0x7f2ec0075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f2ed27fc700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:6808/3070586225,v1:172.21.15.123:6809/3070586225] conn(0x7f2ec0073510 0x7f2ec0075920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac003e30 con 0x7f31b406a3f0 2023-06-09T04:35:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800fe10 con 0x7f1b380c7c00 2023-06-09T04:35:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.205+0000 7f1f21ce0700 1 -- start start 2023-06-09T04:35:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28001110 con 0x7f1b380c7c00 2023-06-09T04:35:23.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1f21ce0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f040037f0 0x7f1f1c0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1f21ce0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f1c14d6d0 0x7f1f1c14cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1f21ce0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f1c1f14e0 0x7f1f1c1f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1f21ce0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c1f1e90 con 0x7f1f040037f0 2023-06-09T04:35:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1f21ce0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0c8a50 con 0x7f1f1c14d6d0 2023-06-09T04:35:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1f21ce0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c14d260 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac005070 con 0x7f31b406a3f0 2023-06-09T04:35:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0053e0 con 0x7f31b406a3f0 2023-06-09T04:35:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f1c1f14e0 0x7f1f1c1f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.206+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b381f1b30 con 0x7f1b380c7c00 2023-06-09T04:35:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f1c1f14e0 0x7f1f1c1f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46612/0 (socket says 172.21.15.92:46612) 2023-06-09T04:35:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 -- 172.21.15.92:0/2677589961 learned_addr learned my addr 172.21.15.92:0/2677589961 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 -- 172.21.15.92:0/2677589961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f1c14d6d0 msgr2=0x7f1f1c14cd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:35:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f1c14d6d0 0x7f1f1c14cd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 -- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f040037f0 msgr2=0x7f1f1c0c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1f040037f0 0x7f1f1c0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1f1bfff700 1 -- 172.21.15.92:0/2677589961 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c14f8f0 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b380c6700 con 0x7f1b380c7c00 2023-06-09T04:35:23.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f2ed2ffd700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2ec006ace0 0x7f2ec006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.208+0000 7fcf52450700 1 Processor -- start 2023-06-09T04:35:23.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.208+0000 7fcf52450700 1 -- start start 2023-06-09T04:35:23.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== osd.5 v2:172.21.15.123:6808/3070586225 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2ec0075fd0 con 0x7f2ec0073510 2023-06-09T04:35:23.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.207+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28025650 con 0x7f1b380c7c00 2023-06-09T04:35:23.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.208+0000 7f2ed2ffd700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2ec006ace0 0x7f2ec006d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8001370 tx=0x7f2ec80010b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b41f6b60 con 0x7f31b406a3f0 2023-06-09T04:35:23.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b41f7120 con 0x7f31b406a3f0 2023-06-09T04:35:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf52450700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcf4c069db0 0x7fcf4c06a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf52450700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4c15c860 con 0x7fcf4c15ff50 2023-06-09T04:35:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf52450700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4c0cf1e0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf52450700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4c0710e0 con 0x7fcf4c069db0 2023-06-09T04:35:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf4b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcf4c15ff50 0x7fcf4c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36042/0 (socket says 172.21.15.92:36042) 2023-06-09T04:35:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf4b7fe700 1 -- 172.21.15.92:0/3309802906 learned_addr learned my addr 172.21.15.92:0/3309802906 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b18000fc0 con 0x7f1b380c7c00 2023-06-09T04:35:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf4bfff700 1 --2- 172.21.15.92:0/3309802906 >> v2:172.21.15.123:3300/0 conn(0x7fcf4c069db0 0x7fcf4c06a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf4affd700 1 -- 172.21.15.92:0/3309802906 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175689844 0 0) 0x7fcf4c15c860 con 0x7fcf4c15ff50 2023-06-09T04:35:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf4affd700 1 -- 172.21.15.92:0/3309802906 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf340035b0 con 0x7fcf4c15ff50 2023-06-09T04:35:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.209+0000 7fcf4affd700 1 -- 172.21.15.92:0/3309802906 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969765294 0 0) 0x7fcf4c0cf1e0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4affd700 1 -- 172.21.15.92:0/3309802906 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf4c15c860 con 0x7fcf4c06deb0 2023-06-09T04:35:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31b40d0370 con 0x7f31b406a3f0 2023-06-09T04:35:23.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4bfff700 1 -- 172.21.15.92:0/3309802906 >> v1:172.21.15.123:6789/0 conn(0x7fcf4c06deb0 legacy=0x7fcf4c0709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4bfff700 1 -- 172.21.15.92:0/3309802906 >> v1:172.21.15.92:6789/0 conn(0x7fcf4c15ff50 legacy=0x7fcf4c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4bfff700 1 -- 172.21.15.92:0/3309802906 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c06a6c0 con 0x7fcf4c069db0 2023-06-09T04:35:23.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4bfff700 1 --2- 172.21.15.92:0/3309802906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c069db0 0x7fcf4c06a180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009840 tx=0x7fcf3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a9a6e153726937 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4affd700 1 -- 172.21.15.92:0/3309802906 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c011830 con 0x7fcf4c069db0 2023-06-09T04:35:23.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7f1f1bfff700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f1c1f14e0 0x7f1f1c1f1950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0c2330 tx=0x7f1f1400bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4affd700 1 -- 172.21.15.92:0/3309802906 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c011990 con 0x7fcf4c069db0 2023-06-09T04:35:23.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7fcf4affd700 1 -- 172.21.15.92:0/3309802906 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c011d20 con 0x7fcf4c069db0 2023-06-09T04:35:23.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7f1f03fff700 1 -- 172.21.15.92:0/2677589961 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400ce60 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7f1f03fff700 1 -- 172.21.15.92:0/2677589961 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f14015470 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7f1f03fff700 1 -- 172.21.15.92:0/2677589961 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f14015820 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.210+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c14fb90 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c150150 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31ac005540 con 0x7f31b406a3f0 2023-06-09T04:35:23.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7fcf52450700 1 -- 172.21.15.92:0/3309802906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c069db0 msgr2=0x7fcf4c06a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7fcf52450700 1 --2- 172.21.15.92:0/3309802906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c069db0 0x7fcf4c06a180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009840 tx=0x7fcf3c009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:23.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7fcf52450700 1 -- 172.21.15.92:0/3309802906 shutdown_connections 2023-06-09T04:35:23.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7fcf52450700 1 --2- 172.21.15.92:0/3309802906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c069db0 0x7fcf4c06a180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7fcf52450700 1 -- 172.21.15.92:0/3309802906 >> 172.21.15.92:0/3309802906 conn(0x7fcf4c0b0ff0 msgr2=0x7fcf4c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7fcf52450700 1 -- 172.21.15.92:0/3309802906 shutdown_connections 2023-06-09T04:35:23.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7fcf52450700 1 -- 172.21.15.92:0/3309802906 wait complete. 2023-06-09T04:35:23.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7f31b0ff9700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31a006ace0 0x7f31a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.211+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31ac089ba0 con 0x7f31b406a3f0 2023-06-09T04:35:23.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.212+0000 7f31b0ff9700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:6824/528556293,v1:172.21.15.123:6825/528556293] conn(0x7f31a0073510 0x7f31a0075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.212+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 --> [v2:172.21.15.123:6824/528556293,v1:172.21.15.123:6825/528556293] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31a0075fd0 con 0x7f31a0073510 2023-06-09T04:35:23.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.212+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31ac089ff0 con 0x7f31b406a3f0 2023-06-09T04:35:23.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.212+0000 7f31b89fe700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:6824/528556293,v1:172.21.15.123:6825/528556293] conn(0x7f31a0073510 0x7f31a0075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.213+0000 7f31b3fff700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31a006ace0 0x7f31a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.213+0000 7fcf52450700 1 Processor -- start 2023-06-09T04:35:23.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.213+0000 7f31b3fff700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31a006ace0 0x7f31a006d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31a4000c00 tx=0x7f31a4008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.213+0000 7f1f03fff700 1 -- 172.21.15.92:0/2677589961 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f14012070 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.213+0000 7f1f03fff700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f0806ad10 0x7f1f0806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.213+0000 7f1f03fff700 1 -- 172.21.15.92:0/2677589961 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f14089340 con 0x7f1f1c1f14e0 2023-06-09T04:35:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.214+0000 7f1f1b7fe700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f0806ad10 0x7f1f0806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.214+0000 7f1f1b7fe700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f0806ad10 0x7f1f0806d1c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00c370 tx=0x7f1f0c00f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7f31b89fe700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:6824/528556293,v1:172.21.15.123:6825/528556293] conn(0x7f31a0073510 0x7f31a0075920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.213+0000 7fcf52450700 1 -- start start 2023-06-09T04:35:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf52450700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf4c069db0 0x7fcf4c0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf52450700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf4c06deb0 0x7fcf4c1f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf52450700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c15ff50 0x7fcf4c1f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf52450700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0c3d40 con 0x7fcf4c069db0 2023-06-09T04:35:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf52450700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c1f21a0 con 0x7fcf4c15ff50 2023-06-09T04:35:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf52450700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0c3130 con 0x7fcf4c06deb0 2023-06-09T04:35:23.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf4c06deb0 0x7fcf4c1f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf4c06deb0 0x7fcf4c1f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46630/0 (socket says 172.21.15.92:46630) 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 -- 172.21.15.92:0/2459609017 learned_addr learned my addr 172.21.15.92:0/2459609017 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf509ed700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c15ff50 0x7fcf4c1f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 -- 172.21.15.92:0/2459609017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c15ff50 msgr2=0x7fcf4c1f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf4c15ff50 0x7fcf4c1f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 -- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf4c069db0 msgr2=0x7fcf4c0afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcf4c069db0 0x7fcf4c0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 -- 172.21.15.92:0/2459609017 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0094a0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7fcf4b7fe700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf4c06deb0 0x7fcf4c1f58f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c4d0 tx=0x7fcf4000c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.215+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== osd.6 v2:172.21.15.123:6824/528556293 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f31a0075fd0 con 0x7f31a0073510 2023-06-09T04:35:23.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.216+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40005d70 con 0x7fcf4c06deb0 2023-06-09T04:35:23.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.216+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf400070d0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.216+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40007480 con 0x7fcf4c06deb0 2023-06-09T04:35:23.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.216+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c0c3410 con 0x7fcf4c06deb0 2023-06-09T04:35:23.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.216+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c0c39d0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.216+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf4c06c3c0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.217+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf400075e0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.217+0000 7fcf48ff9700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf3806ace0 0x7fcf3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.217+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf40088c60 con 0x7fcf4c06deb0 2023-06-09T04:35:23.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b2800a030 con 0x7f1b380c7c00 2023-06-09T04:35:23.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7fcf48ff9700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6816/2565093508,v1:172.21.15.92:6817/2565093508] conn(0x7fcf38073510 0x7fcf38075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 --> [v2:172.21.15.92:6816/2565093508,v1:172.21.15.92:6817/2565093508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf38075fd0 con 0x7fcf38073510 2023-06-09T04:35:23.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf400890b0 con 0x7fcf4c06deb0 2023-06-09T04:35:23.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1f21ce0700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6800/3648230153,v1:172.21.15.92:6801/3648230153] conn(0x7f1f1c159ff0 0x7f1f1c1edf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 --> [v2:172.21.15.92:6800/3648230153,v1:172.21.15.92:6801/3648230153] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f1c06a730 con 0x7f1f1c159ff0 2023-06-09T04:35:23.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1f1affd700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6800/3648230153,v1:172.21.15.92:6801/3648230153] conn(0x7f1f1c159ff0 0x7f1f1c1edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7fcf509ed700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6816/2565093508,v1:172.21.15.92:6817/2565093508] conn(0x7fcf38073510 0x7fcf38075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1b2dffb700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1b1c06ad30 0x7f1b1c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7fcf509ed700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6816/2565093508,v1:172.21.15.92:6817/2565093508] conn(0x7fcf38073510 0x7fcf38075920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b2808ad20 con 0x7f1b380c7c00 2023-06-09T04:35:23.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1f1affd700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6800/3648230153,v1:172.21.15.92:6801/3648230153] conn(0x7f1f1c159ff0 0x7f1f1c1edf60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1b2dffb700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6824/445681078,v1:172.21.15.92:6825/445681078] conn(0x7f1b1c073560 0x7f1b1c075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.218+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 --> [v2:172.21.15.92:6824/445681078,v1:172.21.15.92:6825/445681078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b1c076020 con 0x7f1b1c073560 2023-06-09T04:35:23.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.219+0000 7f1b3cccc700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6824/445681078,v1:172.21.15.92:6825/445681078] conn(0x7f1b1c073560 0x7f1b1c075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.219+0000 7f1b3cccc700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6824/445681078,v1:172.21.15.92:6825/445681078] conn(0x7f1b1c073560 0x7f1b1c075970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.219+0000 7f1b3d4cd700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1b1c06ad30 0x7f1b1c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.219+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== osd.3 v2:172.21.15.92:6816/2565093508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcf38075fd0 con 0x7fcf38073510 2023-06-09T04:35:23.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.220+0000 7f1b3d4cd700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1b1c06ad30 0x7f1b1c06d1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006d20 tx=0x7f1b34006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.220+0000 7fcf4bfff700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf3806ace0 0x7fcf3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.220+0000 7f1f03fff700 1 -- 172.21.15.92:0/2677589961 <== osd.1 v2:172.21.15.92:6800/3648230153 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1f10009600 con 0x7f1f1c159ff0 2023-06-09T04:35:23.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.220+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b28004d70 con 0x7f1b380c7c00 2023-06-09T04:35:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.220+0000 7fcf4bfff700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf3806ace0 0x7fcf3806d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0713f0 tx=0x7fcf3c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.220+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== osd.2 v2:172.21.15.92:6824/445681078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1b1c076020 con 0x7f1b1c073560 2023-06-09T04:35:23.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.229+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 --> [v2:172.21.15.92:6808/3209682425,v1:172.21.15.92:6809/3209682425] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8c016d3c0 con 0x7ff8a4073590 2023-06-09T04:35:23.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.231+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 --> [v2:172.21.15.123:6808/3070586225,v1:172.21.15.123:6809/3070586225] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ed416d590 con 0x7f2ec0073510 2023-06-09T04:35:23.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.231+0000 7ff8b6ffd700 1 -- 172.21.15.92:0/1014612536 <== osd.0 v2:172.21.15.92:6808/3209682425 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff8bc007dc0 con 0x7ff8a4073590 2023-06-09T04:35:23.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.231+0000 7f2ebb7fe700 1 -- 172.21.15.92:0/2253508265 <== osd.5 v2:172.21.15.123:6808/3070586225 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2ec4006820 con 0x7f2ec0073510 2023-06-09T04:35:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.232+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:6808/3070586225,v1:172.21.15.123:6809/3070586225] conn(0x7f2ec0073510 msgr2=0x7f2ec0075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.232+0000 7f2ed960d700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:6808/3070586225,v1:172.21.15.123:6809/3070586225] conn(0x7f2ec0073510 0x7f2ec0075920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.232+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2ec006ace0 msgr2=0x7f2ec006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.232+0000 7f2ed960d700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2ec006ace0 0x7f2ec006d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8001370 tx=0x7f2ec80010b0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.232+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ed416d020 msgr2=0x7f2ed4207750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.232+0000 7f2ed960d700 1 --2- 172.21.15.92:0/2253508265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ed416d020 0x7f2ed4207750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc018040 tx=0x7f2ecc004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7f2ed37fe700 1 -- 172.21.15.92:0/2253508265 reap_dead start 2023-06-09T04:35:23.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.232+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6808/3209682425,v1:172.21.15.92:6809/3209682425] conn(0x7ff8a4073590 msgr2=0x7ff8a40759a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c8681700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6808/3209682425,v1:172.21.15.92:6809/3209682425] conn(0x7ff8a4073590 0x7ff8a40759a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 shutdown_connections 2023-06-09T04:35:23.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 >> 172.21.15.92:0/2253508265 conn(0x7f2ed40b0ff0 msgr2=0x7f2ed4167770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff8a406ace0 msgr2=0x7ff8a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c8681700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff8a406ace0 0x7ff8a406d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8008cc0 tx=0x7ff8b801f040 comp rx=0 tx=0).stop 2023-06-09T04:35:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8c016cbb0 msgr2=0x7ff8c00c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c8681700 1 --2- 172.21.15.92:0/1014612536 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8c016cbb0 0x7ff8c00c4ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000b3e0 tx=0x7ff8b000b6f0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c6c1e700 1 -- 172.21.15.92:0/1014612536 reap_dead start 2023-06-09T04:35:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 shutdown_connections 2023-06-09T04:35:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 >> 172.21.15.92:0/1014612536 conn(0x7ff8c00b0ff0 msgr2=0x7ff8c0168c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 shutdown_connections 2023-06-09T04:35:23.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.233+0000 7f2ed960d700 1 -- 172.21.15.92:0/2253508265 wait complete. 2023-06-09T04:35:23.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.236+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 shutdown_connections 2023-06-09T04:35:23.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.236+0000 7ff8c8681700 1 -- 172.21.15.92:0/1014612536 wait complete. 2023-06-09T04:35:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.236+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 --> [v2:172.21.15.123:6800/909955472,v1:172.21.15.123:6801/909955472] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08880ad790 con 0x7f0874073650 2023-06-09T04:35:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.240+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 --> [v2:172.21.15.123:6824/528556293,v1:172.21.15.123:6825/528556293] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31b40d0090 con 0x7f31a0073510 2023-06-09T04:35:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.240+0000 7f086f7fe700 1 -- 172.21.15.92:0/3751747085 <== osd.4 v2:172.21.15.123:6800/909955472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f088000b2d0 con 0x7f0874073650 2023-06-09T04:35:23.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f31b0ff9700 1 -- 172.21.15.92:0/824531975 <== osd.6 v2:172.21.15.123:6824/528556293 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f31a8001400 con 0x7f31a0073510 2023-06-09T04:35:23.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:6800/909955472,v1:172.21.15.123:6801/909955472] conn(0x7f0874073650 msgr2=0x7f0874075a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f088d28c700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:6800/909955472,v1:172.21.15.123:6801/909955472] conn(0x7f0874073650 0x7f0874075a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f087406adb0 msgr2=0x7f087406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f088d28c700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f087406adb0 0x7f087406d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f087c007fd0 tx=0x7f087c009040 comp rx=0 tx=0).stop 2023-06-09T04:35:23.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08880b1360 msgr2=0x7f08880b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f088d28c700 1 --2- 172.21.15.92:0/3751747085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08880b1360 0x7f08880b2710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f087800e9b0 tx=0x7f087800ecc0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.241+0000 7f08877fe700 1 -- 172.21.15.92:0/3751747085 reap_dead start 2023-06-09T04:35:23.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:6824/528556293,v1:172.21.15.123:6825/528556293] conn(0x7f31a0073510 msgr2=0x7f31a0075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f31ba461700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.123:6824/528556293,v1:172.21.15.123:6825/528556293] conn(0x7f31a0073510 0x7f31a0075920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31a006ace0 msgr2=0x7f31a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f31ba461700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31a006ace0 0x7f31a006d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31a4000c00 tx=0x7f31a4008070 comp rx=0 tx=0).stop 2023-06-09T04:35:23.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31b406a3f0 msgr2=0x7f31b40c4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f31ba461700 1 --2- 172.21.15.92:0/824531975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31b406a3f0 0x7f31b40c4f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0012e0 tx=0x7f31ac00cac0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f31b89fe700 1 -- 172.21.15.92:0/824531975 reap_dead start 2023-06-09T04:35:23.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 --> [v2:172.21.15.92:6824/445681078,v1:172.21.15.92:6825/445681078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b18002d40 con 0x7f1b1c073560 2023-06-09T04:35:23.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f1b2dffb700 1 -- 172.21.15.92:0/1808431036 <== osd.2 v2:172.21.15.92:6824/445681078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1b18002d40 con 0x7f1b1c073560 2023-06-09T04:35:23.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f0e18e81700 1 Processor -- start 2023-06-09T04:35:23.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.242+0000 7f0e18e81700 1 -- start start 2023-06-09T04:35:23.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 shutdown_connections 2023-06-09T04:35:23.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 >> 172.21.15.92:0/3751747085 conn(0x7f0888098c10 msgr2=0x7f088806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f0e18e81700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e14069db0 0x7f0e1406a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f0e18e81700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 0x7f0e1406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f0e18e81700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1406a740 con 0x7f0e14163cf0 2023-06-09T04:35:23.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f0e18e81700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e141640c0 con 0x7f0e14069db0 2023-06-09T04:35:23.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.243+0000 7f0e18e81700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140702c0 con 0x7f0e1406c320 2023-06-09T04:35:23.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e1259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0e14163cf0 0x7f0e140cc400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36066/0 (socket says 172.21.15.92:36066) 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e1259c700 1 -- 172.21.15.92:0/3543342382 learned_addr learned my addr 172.21.15.92:0/3543342382 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e12d9d700 1 --2- 172.21.15.92:0/3543342382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 0x7f0e1406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e11d9b700 1 --2- 172.21.15.92:0/3543342382 >> v2:172.21.15.92:3300/0 conn(0x7f0e14069db0 0x7f0e1406a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e1159a700 1 -- 172.21.15.92:0/3543342382 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236631432 0 0) 0x7f0e04001110 con 0x7f0e14163cf0 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e12d9d700 1 -- 172.21.15.92:0/3543342382 >> v2:172.21.15.92:3300/0 conn(0x7f0e14069db0 msgr2=0x7f0e1406a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e12d9d700 1 --2- 172.21.15.92:0/3543342382 >> v2:172.21.15.92:3300/0 conn(0x7f0e14069db0 0x7f0e1406a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e12d9d700 1 -- 172.21.15.92:0/3543342382 >> v1:172.21.15.92:6789/0 conn(0x7f0e14163cf0 legacy=0x7f0e140cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e12d9d700 1 -- 172.21.15.92:0/3543342382 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140705a0 con 0x7f0e1406c320 2023-06-09T04:35:23.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6824/445681078,v1:172.21.15.92:6825/445681078] conn(0x7f1b1c073560 msgr2=0x7f1b1c075970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f1b3f731700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6824/445681078,v1:172.21.15.92:6825/445681078] conn(0x7f1b1c073560 0x7f1b1c075970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 shutdown_connections 2023-06-09T04:35:23.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 shutdown_connections 2023-06-09T04:35:23.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 >> 172.21.15.92:0/824531975 conn(0x7f31b4098c10 msgr2=0x7f31b40ac400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f0e12d9d700 1 --2- 172.21.15.92:0/3543342382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 0x7f0e1406c7a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009d90 tx=0x7f0e0c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c693d3f58c3541a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.244+0000 7f088d28c700 1 -- 172.21.15.92:0/3751747085 wait complete. 2023-06-09T04:35:23.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.245+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 shutdown_connections 2023-06-09T04:35:23.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.245+0000 7f31ba461700 1 -- 172.21.15.92:0/824531975 wait complete. 2023-06-09T04:35:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.245+0000 7f0e1159a700 1 -- 172.21.15.92:0/3543342382 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00e070 con 0x7f0e1406c320 2023-06-09T04:35:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.245+0000 7f0e1159a700 1 -- 172.21.15.92:0/3543342382 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00bb20 con 0x7f0e1406c320 2023-06-09T04:35:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.245+0000 7f0e1159a700 1 -- 172.21.15.92:0/3543342382 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c012430 con 0x7f0e1406c320 2023-06-09T04:35:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 -- 172.21.15.92:0/3543342382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 msgr2=0x7f0e1406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 --2- 172.21.15.92:0/3543342382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 0x7f0e1406c7a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009d90 tx=0x7f0e0c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 -- 172.21.15.92:0/3543342382 shutdown_connections 2023-06-09T04:35:23.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 --2- 172.21.15.92:0/3543342382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 0x7f0e1406c7a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 --2- 172.21.15.92:0/3543342382 >> v2:172.21.15.92:3300/0 conn(0x7f0e14069db0 0x7f0e1406a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 -- 172.21.15.92:0/3543342382 >> 172.21.15.92:0/3543342382 conn(0x7f0e140b0ff0 msgr2=0x7f0e140ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 -- 172.21.15.92:0/3543342382 shutdown_connections 2023-06-09T04:35:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.246+0000 7f0e18e81700 1 -- 172.21.15.92:0/3543342382 wait complete. 2023-06-09T04:35:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.249+0000 7f0e18e81700 1 Processor -- start 2023-06-09T04:35:23.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.249+0000 7f0e18e81700 1 -- start start 2023-06-09T04:35:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.249+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 --> [v2:172.21.15.92:6800/3648230153,v1:172.21.15.92:6801/3648230153] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f1c071120 con 0x7f1f1c159ff0 2023-06-09T04:35:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.249+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1b1c06ad30 msgr2=0x7f1b1c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.249+0000 7f1b3f731700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1b1c06ad30 0x7f1b1c06d1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b34006d20 tx=0x7f1b34006c90 comp rx=0 tx=0).stop 2023-06-09T04:35:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.249+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b380c7c00 msgr2=0x7f1b380c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.249+0000 7f1b3f731700 1 --2- 172.21.15.92:0/1808431036 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b380c7c00 0x7f1b380c8060 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1b380c2330 tx=0x7f1b28010000 comp rx=0 tx=0).stop 2023-06-09T04:35:23.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f1b3dcce700 1 -- 172.21.15.92:0/1808431036 reap_dead start 2023-06-09T04:35:23.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e18e81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14069db0 0x7f0e1415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e18e81700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 0x7f0e1415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e18e81700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14163cf0 0x7f0e140b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e18e81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1415e630 con 0x7f0e14069db0 2023-06-09T04:35:23.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e18e81700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1415dab0 con 0x7f0e14163cf0 2023-06-09T04:35:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e18e81700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1415dd90 con 0x7f0e1406c320 2023-06-09T04:35:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e1259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14069db0 0x7f0e1415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e1259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14069db0 0x7f0e1415aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37742/0 (socket says 172.21.15.92:37742) 2023-06-09T04:35:23.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f0e1259c700 1 -- 172.21.15.92:0/1379486127 learned_addr learned my addr 172.21.15.92:0/1379486127 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.250+0000 7f1f03fff700 1 -- 172.21.15.92:0/2677589961 <== osd.1 v2:172.21.15.92:6800/3648230153 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1f100099c0 con 0x7f1f1c159ff0 2023-06-09T04:35:23.304 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:23.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-09T04:35:23.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6800/3648230153,v1:172.21.15.92:6801/3648230153] conn(0x7f1f1c159ff0 msgr2=0x7f1f1c1edf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6800/3648230153,v1:172.21.15.92:6801/3648230153] conn(0x7f1f1c159ff0 0x7f1f1c1edf60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f0e1259c700 1 -- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 msgr2=0x7f0e1415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f0e1259c700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e1406c320 0x7f0e1415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f0e1259c700 1 -- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14163cf0 msgr2=0x7f0e140b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f0e1259c700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e14163cf0 0x7f0e140b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f0e1259c700 1 -- 172.21.15.92:0/1379486127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c009a70 con 0x7f0e14069db0 2023-06-09T04:35:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f0806ad10 msgr2=0x7f1f0806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f0806ad10 0x7f1f0806d1c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00c370 tx=0x7f1f0c00f000 comp rx=0 tx=0).stop 2023-06-09T04:35:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f1c1f14e0 msgr2=0x7f1f1c1f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 --2- 172.21.15.92:0/2677589961 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f1c1f14e0 0x7f1f1c1f1950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0c2330 tx=0x7f1f1400bec0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f1bfff700 1 -- 172.21.15.92:0/2677589961 reap_dead start 2023-06-09T04:35:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 shutdown_connections 2023-06-09T04:35:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 >> 172.21.15.92:0/2677589961 conn(0x7f1f1c0b0ff0 msgr2=0x7f1f1c15af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f0e1259c700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14069db0 0x7f0e1415aa90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c550 tx=0x7f0e0400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 shutdown_connections 2023-06-09T04:35:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.251+0000 7f1f21ce0700 1 -- 172.21.15.92:0/2677589961 wait complete. 2023-06-09T04:35:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.252+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 shutdown_connections 2023-06-09T04:35:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.252+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 >> 172.21.15.92:0/1808431036 conn(0x7f1b380b0ff0 msgr2=0x7f1b38157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.252+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 shutdown_connections 2023-06-09T04:35:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.252+0000 7f1b3f731700 1 -- 172.21.15.92:0/1808431036 wait complete. 2023-06-09T04:35:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.253+0000 7f0e02ffd700 1 -- 172.21.15.92:0/1379486127 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04004c80 con 0x7f0e14069db0 2023-06-09T04:35:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.253+0000 7f0e02ffd700 1 -- 172.21.15.92:0/1379486127 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e04004e00 con 0x7f0e14069db0 2023-06-09T04:35:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.253+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1415e070 con 0x7f0e14069db0 2023-06-09T04:35:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.253+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e140b2850 con 0x7f0e14069db0 2023-06-09T04:35:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.253+0000 7f0e02ffd700 1 -- 172.21.15.92:0/1379486127 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040072f0 con 0x7f0e14069db0 2023-06-09T04:35:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.255+0000 7f0e02ffd700 1 -- 172.21.15.92:0/1379486127 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e04007450 con 0x7f0e14069db0 2023-06-09T04:35:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.255+0000 7f0e02ffd700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dfc06ada0 0x7f0dfc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.256+0000 7f0e02ffd700 1 -- 172.21.15.92:0/1379486127 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e04088f30 con 0x7f0e14069db0 2023-06-09T04:35:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.256+0000 7f0e11d9b700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dfc06ada0 0x7f0dfc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.256+0000 7f0e11d9b700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dfc06ada0 0x7f0dfc06d250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e08006ac0 tx=0x7f0e08006a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.312 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:23.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-09T04:35:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.258+0000 7f0e18e81700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:6816/3620631491,v1:172.21.15.123:6817/3620631491] conn(0x7f0df80015e0 0x7f0df8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.258+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 --> [v2:172.21.15.123:6816/3620631491,v1:172.21.15.123:6817/3620631491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0df8006bc0 con 0x7f0df80015e0 2023-06-09T04:35:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.258+0000 7f0e12d9d700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:6816/3620631491,v1:172.21.15.123:6817/3620631491] conn(0x7f0df80015e0 0x7f0df8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.258+0000 7f0e12d9d700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:6816/3620631491,v1:172.21.15.123:6817/3620631491] conn(0x7f0df80015e0 0x7f0df8003a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.259+0000 7f0e02ffd700 1 -- 172.21.15.92:0/1379486127 <== osd.7 v2:172.21.15.123:6816/3620631491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0df8006bc0 con 0x7f0df80015e0 2023-06-09T04:35:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.260+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 --> [v2:172.21.15.92:6816/2565093508,v1:172.21.15.92:6817/2565093508] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf4c160ac0 con 0x7fcf38073510 2023-06-09T04:35:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf48ff9700 1 -- 172.21.15.92:0/2459609017 <== osd.3 v2:172.21.15.92:6816/2565093508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf44001fe0 con 0x7fcf38073510 2023-06-09T04:35:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6816/2565093508,v1:172.21.15.92:6817/2565093508] conn(0x7fcf38073510 msgr2=0x7fcf38075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf52450700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6816/2565093508,v1:172.21.15.92:6817/2565093508] conn(0x7fcf38073510 0x7fcf38075920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf3806ace0 msgr2=0x7fcf3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf52450700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcf3806ace0 0x7fcf3806d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0713f0 tx=0x7fcf3c010000 comp rx=0 tx=0).stop 2023-06-09T04:35:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf4c06deb0 msgr2=0x7fcf4c1f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf52450700 1 --2- 172.21.15.92:0/2459609017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcf4c06deb0 0x7fcf4c1f58f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c4d0 tx=0x7fcf4000c7e0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.262+0000 7fcf509ed700 1 -- 172.21.15.92:0/2459609017 reap_dead start 2023-06-09T04:35:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.264+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 shutdown_connections 2023-06-09T04:35:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.264+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 >> 172.21.15.92:0/2459609017 conn(0x7fcf4c0b0ff0 msgr2=0x7fcf4c0abeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.265+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 shutdown_connections 2023-06-09T04:35:23.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.266+0000 7fcf52450700 1 -- 172.21.15.92:0/2459609017 wait complete. 2023-06-09T04:35:23.317 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:23.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-09T04:35:23.318 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-06-09T04:35:23.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-09T04:35:23.319 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:23.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-09T04:35:23.320 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:23.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-09T04:35:23.322 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:23.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-09T04:35:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.284+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 --> [v2:172.21.15.123:6816/3620631491,v1:172.21.15.123:6817/3620631491] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0df8005c40 con 0x7f0df80015e0 2023-06-09T04:35:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e02ffd700 1 -- 172.21.15.92:0/1379486127 <== osd.7 v2:172.21.15.123:6816/3620631491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0df8005c40 con 0x7f0df80015e0 2023-06-09T04:35:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:6816/3620631491,v1:172.21.15.123:6817/3620631491] conn(0x7f0df80015e0 msgr2=0x7f0df8003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.123:6816/3620631491,v1:172.21.15.123:6817/3620631491] conn(0x7f0df80015e0 0x7f0df8003a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dfc06ada0 msgr2=0x7f0dfc06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dfc06ada0 0x7f0dfc06d250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e08006ac0 tx=0x7f0e08006a50 comp rx=0 tx=0).stop 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14069db0 msgr2=0x7f0e1415aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 --2- 172.21.15.92:0/1379486127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e14069db0 0x7f0e1415aa90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c550 tx=0x7f0e0400c910 comp rx=0 tx=0).stop 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e12d9d700 1 -- 172.21.15.92:0/1379486127 reap_dead start 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 shutdown_connections 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 >> 172.21.15.92:0/1379486127 conn(0x7f0e140b0ff0 msgr2=0x7f0e1406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 shutdown_connections 2023-06-09T04:35:23.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.285+0000 7f0e18e81700 1 -- 172.21.15.92:0/1379486127 wait complete. 2023-06-09T04:35:23.326 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:23.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-09T04:35:23.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.561+0000 7f5ab7bc6700 1 Processor -- start 2023-06-09T04:35:23.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7f5ab7bc6700 1 -- start start 2023-06-09T04:35:23.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7ff18b52e700 1 Processor -- start 2023-06-09T04:35:23.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7ffbef7eb700 1 Processor -- start 2023-06-09T04:35:23.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f5ab7bc6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ab0163cf0 0x7f5ab00cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f5ab7bc6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5ab006a4c0 0x7f5ab006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f5ab7bc6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab00702c0 con 0x7f5ab00cc940 2023-06-09T04:35:23.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f5ab7bc6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab01640c0 con 0x7f5ab006a4c0 2023-06-09T04:35:23.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f5ab7bc6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00705a0 con 0x7f5ab0163cf0 2023-06-09T04:35:23.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f5ab5161700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5ab00cc940 0x7f5ab0069db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36082/0 (socket says 172.21.15.92:36082) 2023-06-09T04:35:23.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f5ab5161700 1 -- 172.21.15.92:0/1309847889 learned_addr learned my addr 172.21.15.92:0/1309847889 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7fbe92356700 1 Processor -- start 2023-06-09T04:35:23.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7fbe92356700 1 -- start start 2023-06-09T04:35:23.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7fbe92356700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7fbe92356700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c06da80 con 0x7fbe8c069db0 2023-06-09T04:35:23.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7fbe92356700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c158840 con 0x7fbe8c0c8e80 2023-06-09T04:35:23.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7fbe92356700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c06df40 con 0x7fbe8c0702c0 2023-06-09T04:35:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7f3f7f664700 1 Processor -- start 2023-06-09T04:35:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.563+0000 7f3f7f664700 1 -- start start 2023-06-09T04:35:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7f664700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f781609a0 0x7f3f78164b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7f664700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f78165050 0x7f3f780cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7f664700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f78164330 con 0x7f3f780cc940 2023-06-09T04:35:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7f664700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7806da40 con 0x7f3f781609a0 2023-06-09T04:35:23.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7f664700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7806dd20 con 0x7f3f78165050 2023-06-09T04:35:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7ffbef7eb700 1 -- start start 2023-06-09T04:35:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbef7eb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe8070cd0 0x7ffbe815c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbef7eb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe806aa10 con 0x7ffbe815cbc0 2023-06-09T04:35:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbef7eb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe806a170 con 0x7ffbe81586f0 2023-06-09T04:35:23.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbef7eb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80ae180 con 0x7ffbe8070cd0 2023-06-09T04:35:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbecd86700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe8070cd0 0x7ffbe815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbed587700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ffbe81586f0 0x7ffbe80c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58594/0 (socket says 172.21.15.92:58594) 2023-06-09T04:35:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbed587700 1 -- 172.21.15.92:0/3595535199 learned_addr learned my addr 172.21.15.92:0/3595535199 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7ff18b52e700 1 -- start start 2023-06-09T04:35:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff18b52e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff18415c430 0x7ff1840b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff18b52e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff18b52e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1840cd540 con 0x7ff1840b4b40 2023-06-09T04:35:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff18b52e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1840b4fc0 con 0x7ff18415c430 2023-06-09T04:35:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff18b52e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840d1200 con 0x7ff1840cedd0 2023-06-09T04:35:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff1892ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff18415c430 0x7ff1840b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7f20b0657700 1 Processor -- start 2023-06-09T04:35:23.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.562+0000 7f20b0657700 1 -- start start 2023-06-09T04:35:23.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20b0657700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a8160860 0x7f20a80ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20b0657700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f20a80caf90 0x7f20a80cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff1892ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff18415c430 0x7ff1840b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37764/0 (socket says 172.21.15.92:37764) 2023-06-09T04:35:23.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff1892ca700 1 -- 172.21.15.92:0/3527413791 learned_addr learned my addr 172.21.15.92:0/3527413791 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20b0657700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a806c260 con 0x7f20a80d0070 2023-06-09T04:35:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20b0657700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a80cb950 con 0x7f20a80caf90 2023-06-09T04:35:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20b0657700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80c42a0 con 0x7f20a8160860 2023-06-09T04:35:23.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7fbe8b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7dc01700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3f780cc940 0x7f3f780c1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58602/0 (socket says 172.21.15.92:58602) 2023-06-09T04:35:23.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7dc01700 1 -- 172.21.15.92:0/13260841 learned_addr learned my addr 172.21.15.92:0/13260841 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984775248 0 0) 0x7f5ab00702c0 con 0x7f5ab00cc940 2023-06-09T04:35:23.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a9c0035b0 con 0x7f5ab00cc940 2023-06-09T04:35:23.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7fbe8b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c06d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46682/0 (socket says 172.21.15.92:46682) 2023-06-09T04:35:23.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1022535531 learned_addr learned my addr 172.21.15.92:0/1022535531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f5ab6163700 1 --2- 172.21.15.92:0/1309847889 >> v2:172.21.15.123:3300/0 conn(0x7f5ab006a4c0 0x7f5ab006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab5962700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ab0163cf0 0x7f5ab00cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1022535531 >> v1:172.21.15.123:6789/0 conn(0x7fbe8c0c8e80 legacy=0x7fbe8c15c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1022535531 >> v1:172.21.15.92:6789/0 conn(0x7fbe8c069db0 legacy=0x7fbe8c06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1022535531 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c070750 con 0x7fbe8c0702c0 2023-06-09T04:35:23.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7fbe8b7fe700 1 --2- 172.21.15.92:0/1022535531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c06d540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e80 tx=0x7fbe8400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39235b4f9c28ed4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f3f7cbff700 1 --2- 172.21.15.92:0/13260841 >> v2:172.21.15.123:3300/0 conn(0x7f3f78165050 0x7f3f780cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f6ffff700 1 -- 172.21.15.92:0/13260841 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403514967 0 0) 0x7f3f78164330 con 0x7f3f780cc940 2023-06-09T04:35:23.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f6ffff700 1 -- 172.21.15.92:0/13260841 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f640035b0 con 0x7f3f780cc940 2023-06-09T04:35:23.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f7cbff700 1 -- 172.21.15.92:0/13260841 >> v2:172.21.15.92:3300/0 conn(0x7f3f781609a0 msgr2=0x7f3f78164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f7cbff700 1 --2- 172.21.15.92:0/13260841 >> v2:172.21.15.92:3300/0 conn(0x7f3f781609a0 0x7f3f78164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f7cbff700 1 -- 172.21.15.92:0/13260841 >> v1:172.21.15.123:6789/0 conn(0x7f3f780cc940 legacy=0x7f3f780c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f6ffff700 1 -- 172.21.15.92:0/13260841 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428791231 0 0) 0x7f3f640035b0 con 0x7f3f780cc940 2023-06-09T04:35:23.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f7cbff700 1 -- 172.21.15.92:0/13260841 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7806e000 con 0x7f3f78165050 2023-06-09T04:35:23.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbecd86700 1 -- 172.21.15.92:0/3595535199 >> v1:172.21.15.123:6789/0 conn(0x7ffbe81586f0 legacy=0x7ffbe80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ffbecd86700 1 -- 172.21.15.92:0/3595535199 >> v1:172.21.15.92:6789/0 conn(0x7ffbe815cbc0 legacy=0x7ffbe80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7ffbecd86700 1 -- 172.21.15.92:0/3595535199 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe806a450 con 0x7ffbe8070cd0 2023-06-09T04:35:23.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7ffbecd86700 1 --2- 172.21.15.92:0/3595535199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe8070cd0 0x7ffbe815c680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009850 tx=0x7ffbe4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b35d59411daad4f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7ff188ac9700 1 --2- 172.21.15.92:0/3527413791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20ae3f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a8160860 0x7f20a80ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20ae3f3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a8160860 0x7f20a80ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46690/0 (socket says 172.21.15.92:46690) 2023-06-09T04:35:23.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20ae3f3700 1 -- 172.21.15.92:0/3871253306 learned_addr learned my addr 172.21.15.92:0/3871253306 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.564+0000 7f20aebf4700 1 --2- 172.21.15.92:0/3871253306 >> v2:172.21.15.123:3300/0 conn(0x7f20a80caf90 0x7f20a80cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f20aebf4700 1 -- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a8160860 msgr2=0x7f20a80ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f20aebf4700 1 --2- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a8160860 0x7f20a80ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f20aebf4700 1 -- 172.21.15.92:0/3871253306 >> v1:172.21.15.92:6789/0 conn(0x7f20a80d0070 legacy=0x7f20a806c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f20aebf4700 1 -- 172.21.15.92:0/3871253306 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a806b970 con 0x7f20a80caf90 2023-06-09T04:35:23.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f20ae3f3700 1 --2- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a8160860 0x7f20a80ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:23.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7ff17bfff700 1 -- 172.21.15.92:0/3527413791 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623394947 0 0) 0x7ff1840cd540 con 0x7ff1840b4b40 2023-06-09T04:35:23.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7ff17bfff700 1 -- 172.21.15.92:0/3527413791 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1700035b0 con 0x7ff1840b4b40 2023-06-09T04:35:23.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464488617 0 0) 0x7f5a9c0035b0 con 0x7f5ab00cc940 2023-06-09T04:35:23.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ab0163cf0 msgr2=0x7f5ab00cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5ab0163cf0 0x7f5ab00cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 >> v2:172.21.15.123:3300/0 conn(0x7f5ab006a4c0 msgr2=0x7f5ab006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 --2- 172.21.15.92:0/1309847889 >> v2:172.21.15.123:3300/0 conn(0x7f5ab006a4c0 0x7f5ab006c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0070880 con 0x7f5ab00cc940 2023-06-09T04:35:23.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f5aa8001a40 con 0x7f5ab00cc940 2023-06-09T04:35:23.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7ffbdffff700 1 -- 172.21.15.92:0/3595535199 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4005e30 con 0x7ffbe8070cd0 2023-06-09T04:35:23.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7ffbdffff700 1 -- 172.21.15.92:0/3595535199 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe4004030 con 0x7ffbe8070cd0 2023-06-09T04:35:23.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7ffbdffff700 1 -- 172.21.15.92:0/3595535199 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe40043c0 con 0x7ffbe8070cd0 2023-06-09T04:35:23.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f3f7cbff700 1 --2- 172.21.15.92:0/13260841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78165050 0x7f3f780cc400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f7806c100 tx=0x7f3f68009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380e779d1b64a72e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f20aebf4700 1 --2- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a80caf90 0x7f20a80cb410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009840 tx=0x7f20a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe03ae38cf63e2a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f20ad3f1700 1 -- 172.21.15.92:0/3871253306 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004230 con 0x7f20a80caf90 2023-06-09T04:35:23.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f20ad3f1700 1 -- 172.21.15.92:0/3871253306 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4004390 con 0x7f20a80caf90 2023-06-09T04:35:23.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f20ad3f1700 1 -- 172.21.15.92:0/3871253306 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40036a0 con 0x7f20a80caf90 2023-06-09T04:35:23.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 >> v1:172.21.15.92:6789/0 conn(0x7f5ab00cc940 legacy=0x7f5ab0069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c003a00 0x7f5a9c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5a9c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a9c005200 0x7f5a9c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.565+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab01640c0 con 0x7f5a9c003a00 2023-06-09T04:35:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00705a0 con 0x7f5a9c005200 2023-06-09T04:35:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c00c050 con 0x7f5a9c004850 2023-06-09T04:35:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f3f6ffff700 1 -- 172.21.15.92:0/13260841 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f700019f0 con 0x7f3f78165050 2023-06-09T04:35:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f3f6ffff700 1 -- 172.21.15.92:0/13260841 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f68004030 con 0x7f3f78165050 2023-06-09T04:35:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f3f6ffff700 1 -- 172.21.15.92:0/13260841 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680043a0 con 0x7f3f78165050 2023-06-09T04:35:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ff188ac9700 1 -- 172.21.15.92:0/3527413791 >> v2:172.21.15.92:3300/0 conn(0x7ff18415c430 msgr2=0x7ff1840b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ff188ac9700 1 --2- 172.21.15.92:0/3527413791 >> v2:172.21.15.92:3300/0 conn(0x7ff18415c430 0x7ff1840b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ff188ac9700 1 -- 172.21.15.92:0/3527413791 >> v1:172.21.15.123:6789/0 conn(0x7ff1840b4b40 legacy=0x7ff1840d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ff188ac9700 1 -- 172.21.15.92:0/3527413791 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840d14e0 con 0x7ff1840cedd0 2023-06-09T04:35:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ff188ac9700 1 --2- 172.21.15.92:0/3527413791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840b4600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff174009840 tx=0x7ff174009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c353917fb99fa633 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ffbef7eb700 1 -- 172.21.15.92:0/3595535199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe8070cd0 msgr2=0x7ffbe815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/3595535199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe8070cd0 0x7ffbe815c680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009850 tx=0x7ffbe4009b60 comp rx=0 tx=0).stop 2023-06-09T04:35:23.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ffbef7eb700 1 -- 172.21.15.92:0/3595535199 shutdown_connections 2023-06-09T04:35:23.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/3595535199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe8070cd0 0x7ffbe815c680 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7ffbef7eb700 1 -- 172.21.15.92:0/3595535199 >> 172.21.15.92:0/3595535199 conn(0x7ffbe80b0ff0 msgr2=0x7ffbe80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f20b0657700 1 -- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a80caf90 msgr2=0x7f20a80cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7f20b0657700 1 --2- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a80caf90 0x7f20a80cb410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009840 tx=0x7f20a4009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:23.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f20b0657700 1 -- 172.21.15.92:0/3871253306 shutdown_connections 2023-06-09T04:35:23.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f20b0657700 1 --2- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a80caf90 0x7f20a80cb410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f20b0657700 1 --2- 172.21.15.92:0/3871253306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a8160860 0x7f20a80ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f20b0657700 1 -- 172.21.15.92:0/3871253306 >> 172.21.15.92:0/3871253306 conn(0x7f20a806e4e0 msgr2=0x7f20a806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f5ab5161700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5a9c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f5ab5161700 1 -- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a9c005200 msgr2=0x7f5a9c00ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T04:35:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7fbe8affd700 1 -- 172.21.15.92:0/1022535531 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400e070 con 0x7fbe8c0702c0 2023-06-09T04:35:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7fbe8affd700 1 -- 172.21.15.92:0/1022535531 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84003a20 con 0x7fbe8c0702c0 2023-06-09T04:35:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.566+0000 7fbe8affd700 1 -- 172.21.15.92:0/1022535531 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84003dd0 con 0x7fbe8c0702c0 2023-06-09T04:35:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7fbe92356700 1 -- 172.21.15.92:0/1022535531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 msgr2=0x7fbe8c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7fbe92356700 1 --2- 172.21.15.92:0/1022535531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c06d540 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e80 tx=0x7fbe8400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f3f7f664700 1 -- 172.21.15.92:0/13260841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78165050 msgr2=0x7f3f780cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f3f7f664700 1 --2- 172.21.15.92:0/13260841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78165050 0x7f3f780cc400 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f7806c100 tx=0x7f3f68009b20 comp rx=0 tx=0).stop 2023-06-09T04:35:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f3f7f664700 1 -- 172.21.15.92:0/13260841 shutdown_connections 2023-06-09T04:35:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f3f7f664700 1 --2- 172.21.15.92:0/13260841 >> v2:172.21.15.92:3300/0 conn(0x7f3f781609a0 0x7f3f78164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f3f7f664700 1 --2- 172.21.15.92:0/13260841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f78165050 0x7f3f780cc400 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f3f7f664700 1 -- 172.21.15.92:0/13260841 >> 172.21.15.92:0/13260841 conn(0x7f3f780b0ff0 msgr2=0x7f3f780ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f3f7f664700 1 -- 172.21.15.92:0/13260841 shutdown_connections 2023-06-09T04:35:23.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7ffbef7eb700 1 -- 172.21.15.92:0/3595535199 shutdown_connections 2023-06-09T04:35:23.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7ffbef7eb700 1 -- 172.21.15.92:0/3595535199 wait complete. 2023-06-09T04:35:23.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f20b0657700 1 -- 172.21.15.92:0/3871253306 shutdown_connections 2023-06-09T04:35:23.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f20b0657700 1 -- 172.21.15.92:0/3871253306 wait complete. 2023-06-09T04:35:23.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f5ab5161700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a9c005200 0x7f5a9c00ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f5ab5161700 1 -- 172.21.15.92:0/1309847889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c003a00 msgr2=0x7f5a9c003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:35:23.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f5ab5161700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a9c003a00 0x7f5a9c003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7f5ab5161700 1 -- 172.21.15.92:0/1309847889 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c0035b0 con 0x7f5a9c004850 2023-06-09T04:35:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7fbe92356700 1 -- 172.21.15.92:0/1022535531 shutdown_connections 2023-06-09T04:35:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7fbe92356700 1 --2- 172.21.15.92:0/1022535531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c06d540 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.567+0000 7fbe92356700 1 -- 172.21.15.92:0/1022535531 >> 172.21.15.92:0/1022535531 conn(0x7fbe8c0b0ff0 msgr2=0x7fbe8c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.568+0000 7fbe92356700 1 -- 172.21.15.92:0/1022535531 shutdown_connections 2023-06-09T04:35:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.568+0000 7fbe92356700 1 -- 172.21.15.92:0/1022535531 wait complete. 2023-06-09T04:35:23.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.568+0000 7ffbef7eb700 1 Processor -- start 2023-06-09T04:35:23.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7f5ab5161700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5a9c004cc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800a7a0 tx=0x7f5a9c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b9e305faeb1f81 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8012600 con 0x7f5a9c004850 2023-06-09T04:35:23.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa8012780 con 0x7f5a9c004850 2023-06-09T04:35:23.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7f5ab4960700 1 -- 172.21.15.92:0/1309847889 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8012b30 con 0x7f5a9c004850 2023-06-09T04:35:23.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7fbe92356700 1 Processor -- start 2023-06-09T04:35:23.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 msgr2=0x7f5a9c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f5ab7bc6700 1 --2- 172.21.15.92:0/1309847889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5a9c004cc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800a7a0 tx=0x7f5a9c005880 comp rx=0 tx=0).stop 2023-06-09T04:35:23.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f5ab6163700 1 -- 172.21.15.92:0/1309847889 reap_dead start 2023-06-09T04:35:23.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7f20b0657700 1 Processor -- start 2023-06-09T04:35:23.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7f20b0657700 1 -- start start 2023-06-09T04:35:23.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1309847889 shutdown_connections 2023-06-09T04:35:23.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1309847889 >> 172.21.15.92:0/1309847889 conn(0x7f5ab00b0ff0 msgr2=0x7f5ab00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.568+0000 7f3f7f664700 1 -- 172.21.15.92:0/13260841 wait complete. 2023-06-09T04:35:23.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f3f7f664700 1 Processor -- start 2023-06-09T04:35:23.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f3f7f664700 1 -- start start 2023-06-09T04:35:23.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1309847889 shutdown_connections 2023-06-09T04:35:23.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7fbe92356700 1 -- start start 2023-06-09T04:35:23.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f243f59e700 1 Processor -- start 2023-06-09T04:35:23.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ff17bfff700 1 -- 172.21.15.92:0/3527413791 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180001670 con 0x7ff1840cedd0 2023-06-09T04:35:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ff17bfff700 1 -- 172.21.15.92:0/3527413791 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174004030 con 0x7ff1840cedd0 2023-06-09T04:35:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f20b0657700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80caf90 0x7f20a80c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7f20b0657700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a80d0070 0x7f20a80cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f20b0657700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a8160860 0x7f20a815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ff17bfff700 1 -- 172.21.15.92:0/3527413791 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1740043c0 con 0x7ff1840cedd0 2023-06-09T04:35:23.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7ff18b52e700 1 -- 172.21.15.92:0/3527413791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 msgr2=0x7ff1840b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.570+0000 7ff18b52e700 1 --2- 172.21.15.92:0/3527413791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840b4600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff174009840 tx=0x7ff174009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:23.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f20b0657700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81f86f0 con 0x7f20a80caf90 2023-06-09T04:35:23.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f20b0657700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81f7840 con 0x7f20a8160860 2023-06-09T04:35:23.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f20b0657700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80c7370 con 0x7f20a80d0070 2023-06-09T04:35:23.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.568+0000 7ffbef7eb700 1 -- start start 2023-06-09T04:35:23.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ffbef7eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbe8070cd0 0x7ffbe815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ffbef7eb700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe81586f0 0x7ffbe8156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ffbef7eb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbe815cbc0 0x7ffbe8158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ffbef7eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe81565a0 con 0x7ffbe8070cd0 2023-06-09T04:35:23.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ffbef7eb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe81fb590 con 0x7ffbe815cbc0 2023-06-09T04:35:23.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.569+0000 7ffbef7eb700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80b1700 con 0x7ffbe81586f0 2023-06-09T04:35:23.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe92356700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c069db0 0x7fbe8c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe92356700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c0ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe92356700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe8c0c8e80 0x7fbe8c0aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe92356700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0c3580 con 0x7fbe8c0c8e80 2023-06-09T04:35:23.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe92356700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0c24e0 con 0x7fbe8c069db0 2023-06-09T04:35:23.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe92356700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0c27c0 con 0x7fbe8c0702c0 2023-06-09T04:35:23.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ff18b52e700 1 -- 172.21.15.92:0/3527413791 shutdown_connections 2023-06-09T04:35:23.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ff18b52e700 1 --2- 172.21.15.92:0/3527413791 >> v2:172.21.15.92:3300/0 conn(0x7ff18415c430 0x7ff1840b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ff18b52e700 1 --2- 172.21.15.92:0/3527413791 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840b4600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ff18b52e700 1 -- 172.21.15.92:0/3527413791 >> 172.21.15.92:0/3527413791 conn(0x7ff1840cf600 msgr2=0x7ff1840cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbecd86700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe81586f0 0x7ffbe8156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbedd88700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbe815cbc0 0x7ffbe8158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbecd86700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe81586f0 0x7ffbe8156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46704/0 (socket says 172.21.15.92:46704) 2023-06-09T04:35:23.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbedd88700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbe815cbc0 0x7ffbe8158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58200/0 (socket says 172.21.15.92:58200) 2023-06-09T04:35:23.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbecd86700 1 -- 172.21.15.92:0/1034134985 learned_addr learned my addr 172.21.15.92:0/1034134985 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe8b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c0ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ff18b52e700 1 -- 172.21.15.92:0/3527413791 shutdown_connections 2023-06-09T04:35:23.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ff18b52e700 1 -- 172.21.15.92:0/3527413791 wait complete. 2023-06-09T04:35:23.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe8b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c0ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46706/0 (socket says 172.21.15.92:46706) 2023-06-09T04:35:23.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1147304303 learned_addr learned my addr 172.21.15.92:0/1147304303 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbed587700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbe8070cd0 0x7ffbe815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbedd88700 1 -- 172.21.15.92:0/1034134985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe81586f0 msgr2=0x7ffbe8156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbedd88700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe81586f0 0x7ffbe8156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbedd88700 1 -- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbe8070cd0 msgr2=0x7ffbe815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbedd88700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbe8070cd0 0x7ffbe815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7ffbedd88700 1 -- 172.21.15.92:0/1034134985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe4009430 con 0x7ffbe815cbc0 2023-06-09T04:35:23.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe8bfff700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c069db0 0x7fbe8c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c069db0 msgr2=0x7fbe8c151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7fbe8b7fe700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c069db0 0x7fbe8c151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe8c0c8e80 msgr2=0x7fbe8c0aec80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:35:23.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe8b7fe700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe8c0c8e80 0x7fbe8c0aec80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe8b7fe700 1 -- 172.21.15.92:0/1147304303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84009b30 con 0x7fbe8c0702c0 2023-06-09T04:35:23.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe8bfff700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c069db0 0x7fbe8c151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:23.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f3f7f664700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f780cc940 0x7f3f780b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f3f7f664700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f781609a0 0x7f3f78160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f3f7f664700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f78165050 0x7f3f78143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f3f7f664700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f781618c0 con 0x7f3f781609a0 2023-06-09T04:35:23.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f3f7f664700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78161ba0 con 0x7f3f780cc940 2023-06-09T04:35:23.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f3f7f664700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f78161e80 con 0x7f3f78165050 2023-06-09T04:35:23.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f780cc940 0x7f3f780b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f780cc940 0x7f3f780b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58208/0 (socket says 172.21.15.92:58208) 2023-06-09T04:35:23.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 -- 172.21.15.92:0/2505234461 learned_addr learned my addr 172.21.15.92:0/2505234461 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7dc01700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f78165050 0x7f3f78143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe8b7fe700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c0ae740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e20 tx=0x7fbe84004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7ffbedd88700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbe815cbc0 0x7ffbe8158ac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000be00 tx=0x7ffbe0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe88ff9700 1 -- 172.21.15.92:0/1147304303 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84004500 con 0x7fbe8c0702c0 2023-06-09T04:35:23.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe88ff9700 1 -- 172.21.15.92:0/1147304303 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84004660 con 0x7fbe8c0702c0 2023-06-09T04:35:23.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7fbe88ff9700 1 -- 172.21.15.92:0/1147304303 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84012430 con 0x7fbe8c0702c0 2023-06-09T04:35:23.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1309847889 wait complete. 2023-06-09T04:35:23.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.571+0000 7f243f59e700 1 -- start start 2023-06-09T04:35:23.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f2440158a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243f59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24400cb5b0 0x7f24400cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243f59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24400b0110 con 0x7f24400ce0d0 2023-06-09T04:35:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243f59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24400b03f0 con 0x7f24400cb5b0 2023-06-09T04:35:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400b06d0 con 0x7f24400cb1e0 2023-06-09T04:35:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24400ce0d0 0x7f24400c9110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36138/0 (socket says 172.21.15.92:36138) 2023-06-09T04:35:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243ed9d700 1 -- 172.21.15.92:0/2186274410 learned_addr learned my addr 172.21.15.92:0/2186274410 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f243dd9b700 1 --2- 172.21.15.92:0/2186274410 >> v2:172.21.15.123:3300/0 conn(0x7f24400cb5b0 0x7f24400cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f243e59c700 1 --2- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f2440158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7cbff700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f781609a0 0x7f3f78160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 -- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f78165050 msgr2=0x7f3f78143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f78165050 0x7f3f78143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 -- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f781609a0 msgr2=0x7f3f78160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f781609a0 0x7f3f78160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f3f7d400700 1 -- 172.21.15.92:0/2505234461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f680094a0 con 0x7f3f780cc940 2023-06-09T04:35:23.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c0c2aa0 con 0x7fbe8c0702c0 2023-06-09T04:35:23.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7ff18b52e700 1 Processor -- start 2023-06-09T04:35:23.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7ff18b52e700 1 -- start start 2023-06-09T04:35:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f20aebf4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a8160860 0x7f20a815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f20aebf4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a8160860 0x7f20a815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58198/0 (socket says 172.21.15.92:58198) 2023-06-09T04:35:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7f20aebf4700 1 -- 172.21.15.92:0/669803576 learned_addr learned my addr 172.21.15.92:0/669803576 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20ae3f3700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80caf90 0x7f20a80c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20adbf2700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a80d0070 0x7f20a80cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20aebf4700 1 -- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a80d0070 msgr2=0x7f20a80cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20aebf4700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a80d0070 0x7f20a80cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20aebf4700 1 -- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80caf90 msgr2=0x7f20a80c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20aebf4700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80caf90 0x7f20a80c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20aebf4700 1 -- 172.21.15.92:0/669803576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a0009b30 con 0x7f20a8160860 2023-06-09T04:35:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f20aebf4700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a8160860 0x7f20a815cdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009b90 tx=0x7f20a4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7ffbddffb700 1 -- 172.21.15.92:0/1034134985 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000cc10 con 0x7ffbe815cbc0 2023-06-09T04:35:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7ffbddffb700 1 -- 172.21.15.92:0/1034134985 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe000cd70 con 0x7ffbe815cbc0 2023-06-09T04:35:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7ffbddffb700 1 -- 172.21.15.92:0/1034134985 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0003c40 con 0x7ffbe815cbc0 2023-06-09T04:35:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.572+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe80b19e0 con 0x7ffbe815cbc0 2023-06-09T04:35:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe80b1fa0 con 0x7ffbe815cbc0 2023-06-09T04:35:23.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7f3f7d400700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f780cc940 0x7f3f780b2ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400b530 tx=0x7f3f7400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f5ab7bc6700 1 Processor -- start 2023-06-09T04:35:23.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f3f6dffb700 1 -- 172.21.15.92:0/2505234461 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74011830 con 0x7f3f780cc940 2023-06-09T04:35:23.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f3f6dffb700 1 -- 172.21.15.92:0/2505234461 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74011990 con 0x7f3f780cc940 2023-06-09T04:35:23.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f3f6dffb700 1 -- 172.21.15.92:0/2505234461 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74011d40 con 0x7f3f780cc940 2023-06-09T04:35:23.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe80b21c0 con 0x7ffbe815cbc0 2023-06-09T04:35:23.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.573+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c0c3060 con 0x7fbe8c0702c0 2023-06-09T04:35:23.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f243dd9b700 1 -- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 msgr2=0x7f2440158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f243dd9b700 1 --2- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f2440158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f243dd9b700 1 -- 172.21.15.92:0/2186274410 >> v1:172.21.15.92:6789/0 conn(0x7f24400ce0d0 legacy=0x7f24400c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f243dd9b700 1 -- 172.21.15.92:0/2186274410 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400b09b0 con 0x7f24400cb5b0 2023-06-09T04:35:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f78162160 con 0x7f3f780cc940 2023-06-09T04:35:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f78163d50 con 0x7f3f780cc940 2023-06-09T04:35:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff18b52e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1840b4b40 0x7ff1840cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff18b52e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff18b52e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff18415c430 0x7ff1840b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff18b52e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840c7880 con 0x7ff1840b4b40 2023-06-09T04:35:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff18b52e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840cc240 con 0x7ff18415c430 2023-06-09T04:35:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff18b52e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840c6c70 con 0x7ff1840cedd0 2023-06-09T04:35:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff1892ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1840b4b40 0x7ff1840cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.574+0000 7ff1892ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1840b4b40 0x7ff1840cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37816/0 (socket says 172.21.15.92:37816) 2023-06-09T04:35:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff1892ca700 1 -- 172.21.15.92:0/1566973216 learned_addr learned my addr 172.21.15.92:0/1566973216 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c0740e0 con 0x7fbe8c0702c0 2023-06-09T04:35:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff1892ca700 1 -- 172.21.15.92:0/1566973216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 msgr2=0x7ff1840c7d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T04:35:23.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff1892ca700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840c7d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff1892ca700 1 -- 172.21.15.92:0/1566973216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff18415c430 msgr2=0x7ff1840b3f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T04:35:23.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff1892ca700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff18415c430 0x7ff1840b3f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff1892ca700 1 -- 172.21.15.92:0/1566973216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1740094a0 con 0x7ff1840b4b40 2023-06-09T04:35:23.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff1892ca700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1840b4b40 0x7ff1840cbd00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00efc0 tx=0x7ff17c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff179ffb700 1 -- 172.21.15.92:0/1566973216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00cbb0 con 0x7ff1840b4b40 2023-06-09T04:35:23.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff179ffb700 1 -- 172.21.15.92:0/1566973216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c00f040 con 0x7ff1840b4b40 2023-06-09T04:35:23.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7ff179ffb700 1 -- 172.21.15.92:0/1566973216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c009e90 con 0x7ff1840b4b40 2023-06-09T04:35:23.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7fbe88ff9700 1 -- 172.21.15.92:0/1147304303 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe84019070 con 0x7fbe8c0702c0 2023-06-09T04:35:23.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7fbe88ff9700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbe7806ace0 0x7fbe7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.576+0000 7fbe88ff9700 1 -- 172.21.15.92:0/1147304303 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe8408a680 con 0x7fbe8c0702c0 2023-06-09T04:35:23.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.576+0000 7fbe8bfff700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbe7806ace0 0x7fbe7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.576+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f780c36f0 con 0x7f3f780cc940 2023-06-09T04:35:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.576+0000 7fbe8bfff700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbe7806ace0 0x7fbe7806d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c000f60 tx=0x7fbe7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.576+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1840c6f50 con 0x7ff1840b4b40 2023-06-09T04:35:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.577+0000 7f243dd9b700 1 --2- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 0x7f24400cdb90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2434009840 tx=0x7f2434009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2755e8fd550f7bc7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7f209effd700 1 -- 172.21.15.92:0/669803576 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4003d60 con 0x7f20a8160860 2023-06-09T04:35:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7f209effd700 1 -- 172.21.15.92:0/669803576 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4011630 con 0x7f20a8160860 2023-06-09T04:35:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7f209effd700 1 -- 172.21.15.92:0/669803576 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40119a0 con 0x7f20a8160860 2023-06-09T04:35:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a40094a0 con 0x7f20a8160860 2023-06-09T04:35:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.575+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a81f7d50 con 0x7f20a8160860 2023-06-09T04:35:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.577+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1840c7510 con 0x7ff1840b4b40 2023-06-09T04:35:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.576+0000 7ffbddffb700 1 -- 172.21.15.92:0/1034134985 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffbe0014440 con 0x7ffbe815cbc0 2023-06-09T04:35:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.577+0000 7f5ab7bc6700 1 -- start start 2023-06-09T04:35:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.577+0000 7ffbddffb700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffbcc06ad30 0x7ffbcc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.577+0000 7ffbed587700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffbcc06ad30 0x7ffbcc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.577+0000 7ffbddffb700 1 -- 172.21.15.92:0/1034134985 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffbe0089980 con 0x7ffbe815cbc0 2023-06-09T04:35:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.577+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20900052f0 con 0x7f20a8160860 2023-06-09T04:35:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7ffbed587700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffbcc06ad30 0x7ffbcc06d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8004930 tx=0x7ffbd8004f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab7bc6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5ab01f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab7bc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ab015c7b0 0x7f5ab01604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab7bc6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab015cc20 0x7f5ab00aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab7bc6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00b2990 con 0x7f5ab015c7b0 2023-06-09T04:35:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab7bc6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00b2c70 con 0x7f5ab015cc20 2023-06-09T04:35:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab7bc6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00b2f50 con 0x7f5a9c004850 2023-06-09T04:35:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ab015c7b0 0x7f5ab01604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5962700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5ab01f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ab015c7b0 0x7f5ab01604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37828/0 (socket says 172.21.15.92:37828) 2023-06-09T04:35:23.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 -- 172.21.15.92:0/1418836417 learned_addr learned my addr 172.21.15.92:0/1418836417 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5962700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5ab01f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46736/0 (socket says 172.21.15.92:46736) 2023-06-09T04:35:23.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab6163700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab015cc20 0x7f5ab00aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7ff179ffb700 1 -- 172.21.15.92:0/1566973216 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff17c014480 con 0x7ff1840b4b40 2023-06-09T04:35:23.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 -- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 msgr2=0x7f5ab01f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5ab01f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 -- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab015cc20 msgr2=0x7f5ab00aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab015cc20 0x7f5ab00aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5161700 1 -- 172.21.15.92:0/1418836417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab00b3230 con 0x7f5ab015c7b0 2023-06-09T04:35:23.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7f5ab5962700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5ab01f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.579+0000 7f5ab5161700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ab015c7b0 0x7f5ab01604b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800ad90 tx=0x7f5aa8001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.578+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1840c7b60 con 0x7ff1840b4b40 2023-06-09T04:35:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.579+0000 7ff179ffb700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff16806ad30 0x7ff16806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.579+0000 7ff179ffb700 1 -- 172.21.15.92:0/1566973216 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff17c089950 con 0x7ff1840b4b40 2023-06-09T04:35:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.579+0000 7ff188ac9700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff16806ad30 0x7ff16806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.579+0000 7ff188ac9700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff16806ad30 0x7ff16806d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff18415b450 tx=0x7ff174008000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.580+0000 7f243d59a700 1 -- 172.21.15.92:0/2186274410 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434004230 con 0x7f24400cb5b0 2023-06-09T04:35:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.580+0000 7ffbddffb700 1 -- 172.21.15.92:0/1034134985 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffbe0054b10 con 0x7ffbe815cbc0 2023-06-09T04:35:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.580+0000 7f243d59a700 1 -- 172.21.15.92:0/2186274410 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434004390 con 0x7f24400cb5b0 2023-06-09T04:35:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.580+0000 7f243d59a700 1 -- 172.21.15.92:0/2186274410 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340036a0 con 0x7f24400cb5b0 2023-06-09T04:35:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 -- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 msgr2=0x7f24400cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 --2- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 0x7f24400cdb90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2434009840 tx=0x7f2434009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:23.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f689480e700 1 Processor -- start 2023-06-09T04:35:23.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 -- 172.21.15.92:0/2186274410 shutdown_connections 2023-06-09T04:35:23.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 --2- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f2440158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 --2- 172.21.15.92:0/2186274410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 0x7f24400cdb90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 -- 172.21.15.92:0/2186274410 >> 172.21.15.92:0/2186274410 conn(0x7f244006c0d0 msgr2=0x7f244006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f3f6dffb700 1 -- 172.21.15.92:0/2505234461 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f7401a400 con 0x7f3f780cc940 2023-06-09T04:35:23.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 -- 172.21.15.92:0/2186274410 shutdown_connections 2023-06-09T04:35:23.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f243f59e700 1 -- 172.21.15.92:0/2186274410 wait complete. 2023-06-09T04:35:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.581+0000 7f689480e700 1 -- start start 2023-06-09T04:35:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.582+0000 7f209effd700 1 -- 172.21.15.92:0/669803576 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20a4018050 con 0x7f20a8160860 2023-06-09T04:35:23.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f209effd700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f208c06adb0 0x7f208c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f689480e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f689015c430 0x7f68900b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f689480e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900cedd0 0x7f68900b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f689480e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68900cd540 con 0x7f68900b4b40 2023-06-09T04:35:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f689480e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900b4fc0 con 0x7f689015c430 2023-06-09T04:35:23.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f689480e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900d1200 con 0x7f68900cedd0 2023-06-09T04:35:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f209effd700 1 -- 172.21.15.92:0/669803576 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20a408a420 con 0x7f20a8160860 2023-06-09T04:35:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.582+0000 7f3f6dffb700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06adb0 0x7f3f5c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.582+0000 7f3f6dffb700 1 -- 172.21.15.92:0/2505234461 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f740899c0 con 0x7f3f780cc940 2023-06-09T04:35:23.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7fbe88ff9700 1 -- 172.21.15.92:0/1147304303 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe840557a0 con 0x7fbe8c0702c0 2023-06-09T04:35:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688e59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f689015c430 0x7f68900b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688dd9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900cedd0 0x7f68900b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688ed9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f68900b4b40 0x7f68900d1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58670/0 (socket says 172.21.15.92:58670) 2023-06-09T04:35:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688ed9d700 1 -- 172.21.15.92:0/1037031684 learned_addr learned my addr 172.21.15.92:0/1037031684 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:23.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f243f59e700 1 Processor -- start 2023-06-09T04:35:23.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688e59c700 1 -- 172.21.15.92:0/1037031684 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900cedd0 msgr2=0x7f68900b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:23.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688e59c700 1 --2- 172.21.15.92:0/1037031684 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900cedd0 0x7f68900b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688e59c700 1 -- 172.21.15.92:0/1037031684 >> v1:172.21.15.123:6789/0 conn(0x7f68900b4b40 legacy=0x7f68900d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f688e59c700 1 -- 172.21.15.92:0/1037031684 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900d14e0 con 0x7f689015c430 2023-06-09T04:35:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f3f7cbff700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06adb0 0x7f3f5c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f20ae3f3700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f208c06adb0 0x7f208c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f688e59c700 1 --2- 172.21.15.92:0/1037031684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b59d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6880009840 tx=0x7f6880009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d651309182a3ea server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.583+0000 7f243f59e700 1 -- start start 2023-06-09T04:35:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f20ae3f3700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f208c06adb0 0x7f208c06d260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20a80b65e0 tx=0x7f20a000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f3f7cbff700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06adb0 0x7f3f5c06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f680090b0 tx=0x7f3f68008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f243f59e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f24401f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f243f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 0x7f24401f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f243f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400ce0d0 0x7f24400b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f243f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401fa3a0 con 0x7f24400ce0d0 2023-06-09T04:35:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f243f59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401f86c0 con 0x7f24400cb5b0 2023-06-09T04:35:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.584+0000 7f243f59e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440154420 con 0x7f24400cb1e0 2023-06-09T04:35:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f24401f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f24401f9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46754/0 (socket says 172.21.15.92:46754) 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 -- 172.21.15.92:0/995563824 learned_addr learned my addr 172.21.15.92:0/995563824 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243dd9b700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 0x7f24401f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243ed9d700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400ce0d0 0x7f24400b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 -- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 msgr2=0x7f24401f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 0x7f24401f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 -- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400ce0d0 msgr2=0x7f24400b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400ce0d0 0x7f24400b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 -- 172.21.15.92:0/995563824 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24340094a0 con 0x7f24400cb1e0 2023-06-09T04:35:23.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243e59c700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f24401f9e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9b0 tx=0x7f243000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f242effd700 1 -- 172.21.15.92:0/995563824 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430009710 con 0x7f24400cb1e0 2023-06-09T04:35:23.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f242effd700 1 -- 172.21.15.92:0/995563824 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243000ee50 con 0x7f24400cb1e0 2023-06-09T04:35:23.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24401f7e40 con 0x7f24400cb1e0 2023-06-09T04:35:23.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f242effd700 1 -- 172.21.15.92:0/995563824 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243000ca50 con 0x7f24400cb1e0 2023-06-09T04:35:23.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401549a0 con 0x7f24400cb1e0 2023-06-09T04:35:23.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.585+0000 7f3f6dffb700 1 -- 172.21.15.92:0/2505234461 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f74054a50 con 0x7f3f780cc940 2023-06-09T04:35:23.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.586+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244004e8e0 con 0x7f24400cb1e0 2023-06-09T04:35:23.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.587+0000 7f5aa67fc700 1 -- 172.21.15.92:0/1418836417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80128f0 con 0x7f5ab015c7b0 2023-06-09T04:35:23.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.587+0000 7ff179ffb700 1 -- 172.21.15.92:0/1566973216 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff17c054a50 con 0x7ff1840b4b40 2023-06-09T04:35:23.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.587+0000 7f5aa67fc700 1 -- 172.21.15.92:0/1418836417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa801f620 con 0x7f5ab015c7b0 2023-06-09T04:35:23.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.587+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab00b3510 con 0x7f5ab015c7b0 2023-06-09T04:35:23.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.587+0000 7f5aa67fc700 1 -- 172.21.15.92:0/1418836417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa801f9d0 con 0x7f5ab015c7b0 2023-06-09T04:35:23.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.587+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab015d190 con 0x7f5ab015c7b0 2023-06-09T04:35:23.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.588+0000 7f242effd700 1 -- 172.21.15.92:0/995563824 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f243000cbb0 con 0x7f24400cb1e0 2023-06-09T04:35:23.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.589+0000 7f242effd700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f242006ace0 0x7f242006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.589+0000 7f242effd700 1 -- 172.21.15.92:0/995563824 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f24300894e0 con 0x7f24400cb1e0 2023-06-09T04:35:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.588+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab00b3880 con 0x7f5ab015c7b0 2023-06-09T04:35:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.589+0000 7f5aa67fc700 1 -- 172.21.15.92:0/1418836417 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5aa801fb30 con 0x7f5ab015c7b0 2023-06-09T04:35:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.590+0000 7f5aa67fc700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a9406ace0 0x7f5a9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.590+0000 7f5aa67fc700 1 -- 172.21.15.92:0/1418836417 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5aa808b930 con 0x7f5ab015c7b0 2023-06-09T04:35:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.592+0000 7f209effd700 1 -- 172.21.15.92:0/669803576 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f20a4055540 con 0x7f20a8160860 2023-06-09T04:35:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.592+0000 7f242effd700 1 -- 172.21.15.92:0/995563824 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2430058010 con 0x7f24400cb1e0 2023-06-09T04:35:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.593+0000 7f688d59a700 1 -- 172.21.15.92:0/1037031684 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880011830 con 0x7f689015c430 2023-06-09T04:35:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.593+0000 7f688d59a700 1 -- 172.21.15.92:0/1037031684 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880011990 con 0x7f689015c430 2023-06-09T04:35:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.593+0000 7f688d59a700 1 -- 172.21.15.92:0/1037031684 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880011d20 con 0x7f689015c430 2023-06-09T04:35:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.593+0000 7f5ab5962700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a9406ace0 0x7f5a9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.593+0000 7f5ab5962700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a9406ace0 0x7f5a9406d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006b20 tx=0x7f5aac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.595+0000 7f243dd9b700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f242006ace0 0x7f242006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.595+0000 7f243dd9b700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f242006ace0 0x7f242006d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2434009840 tx=0x7f2434008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.595+0000 7f689480e700 1 -- 172.21.15.92:0/1037031684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 msgr2=0x7f68900b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.595+0000 7f689480e700 1 --2- 172.21.15.92:0/1037031684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b59d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6880009840 tx=0x7f6880009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.596+0000 7f689480e700 1 -- 172.21.15.92:0/1037031684 shutdown_connections 2023-06-09T04:35:23.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.596+0000 7f689480e700 1 --2- 172.21.15.92:0/1037031684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b59d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.596+0000 7f689480e700 1 --2- 172.21.15.92:0/1037031684 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900cedd0 0x7f68900b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.596+0000 7f689480e700 1 -- 172.21.15.92:0/1037031684 >> 172.21.15.92:0/1037031684 conn(0x7f68900cf600 msgr2=0x7f68900cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.598+0000 7f689480e700 1 -- 172.21.15.92:0/1037031684 shutdown_connections 2023-06-09T04:35:23.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.599+0000 7f5aa67fc700 1 -- 172.21.15.92:0/1418836417 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5aa80569d0 con 0x7f5ab015c7b0 2023-06-09T04:35:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.598+0000 7f689480e700 1 -- 172.21.15.92:0/1037031684 wait complete. 2023-06-09T04:35:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.600+0000 7f689480e700 1 Processor -- start 2023-06-09T04:35:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.600+0000 7f689480e700 1 -- start start 2023-06-09T04:35:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.601+0000 7f689480e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900b4b40 0x7f68901fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.601+0000 7f689480e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68900cedd0 0x7f689014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.601+0000 7f689480e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.601+0000 7f689480e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689015e910 con 0x7f68900cedd0 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.601+0000 7f689480e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689015dfb0 con 0x7f689015c430 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.601+0000 7f689480e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689015e250 con 0x7f68900b4b40 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58262/0 (socket says 172.21.15.92:58262) 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688ed9d700 1 -- 172.21.15.92:0/341932360 learned_addr learned my addr 172.21.15.92:0/341932360 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688e59c700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900b4b40 0x7f68901fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688e59c700 1 -- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 msgr2=0x7f68900b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688e59c700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688e59c700 1 -- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68900cedd0 msgr2=0x7f689014fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.602+0000 7f688dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68900cedd0 0x7f689014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.607+0000 7f688e59c700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68900cedd0 0x7f689014fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.607+0000 7f688e59c700 1 -- 172.21.15.92:0/341932360 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68800094a0 con 0x7f68900b4b40 2023-06-09T04:35:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.607+0000 7f688ed9d700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.607+0000 7f688dd9b700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68900cedd0 0x7f689014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:23.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.608+0000 7f688e59c700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900b4b40 0x7f68901fe9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6880016000 tx=0x7f6880004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.611+0000 7f687effd700 1 -- 172.21.15.92:0/341932360 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880004590 con 0x7f68900b4b40 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.611+0000 7f687effd700 1 -- 172.21.15.92:0/341932360 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688001e070 con 0x7f68900b4b40 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.611+0000 7f687effd700 1 -- 172.21.15.92:0/341932360 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800038e0 con 0x7f68900b4b40 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.612+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689015e530 con 0x7f68900b4b40 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.612+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68900cceb0 con 0x7f68900b4b40 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.614+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68900c94f0 con 0x7f68900b4b40 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.615+0000 7f687effd700 1 -- 172.21.15.92:0/341932360 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6880003a40 con 0x7f68900b4b40 2023-06-09T04:35:23.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.616+0000 7f687effd700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f687006adb0 0x7f687006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.616+0000 7f687effd700 1 -- 172.21.15.92:0/341932360 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f688008a7f0 con 0x7f68900b4b40 2023-06-09T04:35:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.624+0000 7f687effd700 1 -- 172.21.15.92:0/341932360 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6880055910 con 0x7f68900b4b40 2023-06-09T04:35:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.625+0000 7f688dd9b700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f687006adb0 0x7f687006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:23.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.625+0000 7f688dd9b700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f687006adb0 0x7f687006d260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6890152b60 tx=0x7f6884008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:23.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.965+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffbe80740e0 con 0x7ffbe815cbc0 2023-06-09T04:35:23.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.965+0000 7ffbddffb700 1 -- 172.21.15.92:0/1034134985 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffbe0022090 con 0x7ffbe815cbc0 2023-06-09T04:35:23.968 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-06-09T04:35:23.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffbcc06ad30 msgr2=0x7ffbcc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffbcc06ad30 0x7ffbcc06d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8004930 tx=0x7ffbd8004f90 comp rx=0 tx=0).stop 2023-06-09T04:35:23.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbe815cbc0 msgr2=0x7ffbe8158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbe815cbc0 0x7ffbe8158ac0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000be00 tx=0x7ffbe0009ed0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 shutdown_connections 2023-06-09T04:35:23.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ffbcc06ad30 0x7ffbcc06d1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbe815cbc0 0x7ffbe8158ac0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffbe8070cd0 0x7ffbe815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 --2- 172.21.15.92:0/1034134985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffbe81586f0 0x7ffbe8156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.969+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 >> 172.21.15.92:0/1034134985 conn(0x7ffbe80b0ff0 msgr2=0x7ffbe80c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.970+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 shutdown_connections 2023-06-09T04:35:23.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.970+0000 7ffbef7eb700 1 -- 172.21.15.92:0/1034134985 wait complete. 2023-06-09T04:35:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.971+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbe8c0cf0f0 con 0x7fbe8c0702c0 2023-06-09T04:35:23.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.972+0000 7fbe88ff9700 1 -- 172.21.15.92:0/1147304303 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe84058dc0 con 0x7fbe8c0702c0 2023-06-09T04:35:23.975 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-06-09T04:35:23.975 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:23.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.973+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3f780c36f0 con 0x7f3f780cc940 2023-06-09T04:35:23.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.973+0000 7f3f6dffb700 1 -- 172.21.15.92:0/2505234461 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f780c36f0 con 0x7f3f780cc940 2023-06-09T04:35:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbe7806ace0 msgr2=0x7fbe7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7fbe92356700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbe7806ace0 0x7fbe7806d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c000f60 tx=0x7fbe7c009040 comp rx=0 tx=0).stop 2023-06-09T04:35:23.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 msgr2=0x7fbe8c0ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7fbe92356700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c0ae740 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009e20 tx=0x7fbe84004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06adb0 msgr2=0x7f3f5c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06adb0 0x7f3f5c06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f680090b0 tx=0x7f3f68008030 comp rx=0 tx=0).stop 2023-06-09T04:35:23.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f780cc940 msgr2=0x7f3f780b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f780cc940 0x7f3f780b2ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400b530 tx=0x7f3f7400b8f0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 shutdown_connections 2023-06-09T04:35:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06adb0 0x7f3f5c06d260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f78165050 0x7f3f78143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f780cc940 0x7f3f780b2ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 --2- 172.21.15.92:0/2505234461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f781609a0 0x7f3f78160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.977+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 >> 172.21.15.92:0/2505234461 conn(0x7f3f780b0ff0 msgr2=0x7f3f780701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 shutdown_connections 2023-06-09T04:35:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7fbe92356700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbe7806ace0 0x7fbe7806d190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7fbe92356700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe8c0c8e80 0x7fbe8c0aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7fbe92356700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe8c069db0 0x7fbe8c151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7fbe92356700 1 --2- 172.21.15.92:0/1147304303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe8c0702c0 0x7fbe8c0ae740 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 >> 172.21.15.92:0/1147304303 conn(0x7fbe8c0b0ff0 msgr2=0x7fbe8c0ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 shutdown_connections 2023-06-09T04:35:23.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.978+0000 7f3f7f664700 1 -- 172.21.15.92:0/2505234461 wait complete. 2023-06-09T04:35:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.982+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff1840740e0 con 0x7ff1840b4b40 2023-06-09T04:35:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.982+0000 7ff179ffb700 1 -- 172.21.15.92:0/1566973216 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff17c058070 con 0x7ff1840b4b40 2023-06-09T04:35:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.982+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 shutdown_connections 2023-06-09T04:35:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.982+0000 7fbe92356700 1 -- 172.21.15.92:0/1147304303 wait complete. 2023-06-09T04:35:23.983 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.987+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2090005870 con 0x7f20a8160860 2023-06-09T04:35:23.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.988+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff16806ad30 msgr2=0x7ff16806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.988+0000 7ff18b52e700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff16806ad30 0x7ff16806d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff18415b450 tx=0x7ff174008000 comp rx=0 tx=0).stop 2023-06-09T04:35:23.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.988+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1840b4b40 msgr2=0x7ff1840cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.988+0000 7ff18b52e700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1840b4b40 0x7ff1840cbd00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00efc0 tx=0x7ff17c00c3a0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.989+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 shutdown_connections 2023-06-09T04:35:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.989+0000 7ff18b52e700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff16806ad30 0x7ff16806d1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.989+0000 7ff18b52e700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff18415c430 0x7ff1840b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.989+0000 7ff18b52e700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1840b4b40 0x7ff1840cbd00 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00efc0 tx=0x7ff17c00c3a0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.989+0000 7ff18b52e700 1 --2- 172.21.15.92:0/1566973216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff1840cedd0 0x7ff1840c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.989+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 >> 172.21.15.92:0/1566973216 conn(0x7ff1840cf600 msgr2=0x7ff18415a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:23.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-06-09T04:35:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.991+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 shutdown_connections 2023-06-09T04:35:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.991+0000 7ff18b52e700 1 -- 172.21.15.92:0/1566973216 wait complete. 2023-06-09T04:35:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.992+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f24400add00 con 0x7f24400cb1e0 2023-06-09T04:35:23.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.992+0000 7f242effd700 1 -- 172.21.15.92:0/995563824 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f243000ce60 con 0x7f24400cb1e0 2023-06-09T04:35:23.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.993+0000 7f209effd700 1 -- 172.21.15.92:0/669803576 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20a4016090 con 0x7f20a8160860 2023-06-09T04:35:23.994 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:23.997 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.998+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f242006ace0 msgr2=0x7f242006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.998+0000 7f243f59e700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f242006ace0 0x7f242006d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2434009840 tx=0x7f2434008030 comp rx=0 tx=0).stop 2023-06-09T04:35:23.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.998+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 msgr2=0x7f24401f9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:23.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.998+0000 7f243f59e700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f24401f9e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f243000e9b0 tx=0x7f243000ecc0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 shutdown_connections 2023-06-09T04:35:23.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f242006ace0 0x7f242006d190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:23.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24400ce0d0 0x7f24400b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24400cb1e0 0x7f24401f9e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 --2- 172.21.15.92:0/995563824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24400cb5b0 0x7f24401f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 >> 172.21.15.92:0/995563824 conn(0x7f244006c0d0 msgr2=0x7f2440157110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 shutdown_connections 2023-06-09T04:35:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:23.999+0000 7f243f59e700 1 -- 172.21.15.92:0/995563824 wait complete. 2023-06-09T04:35:24.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.000+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f208c06adb0 msgr2=0x7f208c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:24.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.000+0000 7f20b0657700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f208c06adb0 0x7f208c06d260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20a80b65e0 tx=0x7f20a000b040 comp rx=0 tx=0).stop 2023-06-09T04:35:24.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.000+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a8160860 msgr2=0x7f20a815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:24.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.000+0000 7f20b0657700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a8160860 0x7f20a815cdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009b90 tx=0x7f20a4004750 comp rx=0 tx=0).stop 2023-06-09T04:35:24.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.001+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5ab015d3b0 con 0x7f5ab015c7b0 2023-06-09T04:35:24.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.002+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 shutdown_connections 2023-06-09T04:35:24.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.002+0000 7f20b0657700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f208c06adb0 0x7f208c06d260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.002+0000 7f20b0657700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20a8160860 0x7f20a815cdc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.002+0000 7f20b0657700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a80caf90 0x7f20a80c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.002+0000 7f20b0657700 1 --2- 172.21.15.92:0/669803576 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20a80d0070 0x7f20a80cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.002+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 >> 172.21.15.92:0/669803576 conn(0x7f20a806e4e0 msgr2=0x7f20a806c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:24.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-06-09T04:35:24.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.004+0000 7f5aa67fc700 1 -- 172.21.15.92:0/1418836417 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ab015d3b0 con 0x7f5ab015c7b0 2023-06-09T04:35:24.004 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-06-09T04:35:24.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.006+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 shutdown_connections 2023-06-09T04:35:24.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-06-09T04:35:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a9406ace0 msgr2=0x7f5a9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a9406ace0 0x7f5a9406d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5aac006b20 tx=0x7f5aac008040 comp rx=0 tx=0).stop 2023-06-09T04:35:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ab015c7b0 msgr2=0x7f5ab01604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ab015c7b0 0x7f5ab01604b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800ad90 tx=0x7f5aa8001d00 comp rx=0 tx=0).stop 2023-06-09T04:35:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 shutdown_connections 2023-06-09T04:35:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a9406ace0 0x7f5a9406d190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab015cc20 0x7f5ab00aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a9c004850 0x7f5ab01f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 --2- 172.21.15.92:0/1418836417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ab015c7b0 0x7f5ab01604b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 >> 172.21.15.92:0/1418836417 conn(0x7f5ab00b0ff0 msgr2=0x7f5ab00c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f20b0657700 1 -- 172.21.15.92:0/669803576 wait complete. 2023-06-09T04:35:24.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.008+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f689015e390 con 0x7f68900b4b40 2023-06-09T04:35:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.009+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 shutdown_connections 2023-06-09T04:35:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.009+0000 7f5ab7bc6700 1 -- 172.21.15.92:0/1418836417 wait complete. 2023-06-09T04:35:24.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.009+0000 7f687effd700 1 -- 172.21.15.92:0/341932360 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f689015e390 con 0x7f68900b4b40 2023-06-09T04:35:24.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-06-09T04:35:24.012 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f687006adb0 msgr2=0x7f687006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f687006adb0 0x7f687006d260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6890152b60 tx=0x7f6884008040 comp rx=0 tx=0).stop 2023-06-09T04:35:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900b4b40 msgr2=0x7f68901fe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900b4b40 0x7f68901fe9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6880016000 tx=0x7f6880004750 comp rx=0 tx=0).stop 2023-06-09T04:35:24.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 shutdown_connections 2023-06-09T04:35:24.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f687006adb0 0x7f687006d260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f689015c430 0x7f68900b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f68900b4b40 0x7f68901fe9b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 --2- 172.21.15.92:0/341932360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68900cedd0 0x7f689014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:24.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 >> 172.21.15.92:0/341932360 conn(0x7f68900cf600 msgr2=0x7f68900b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:24.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 shutdown_connections 2023-06-09T04:35:24.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:24.011+0000 7f689480e700 1 -- 172.21.15.92:0/341932360 wait complete. 2023-06-09T04:35:24.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-06-09T04:35:24.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-06-09T04:35:24.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-06-09T04:35:24.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-06-09T04:35:24.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-09T04:35:25.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-09T04:35:25.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-09T04:35:25.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-09T04:35:25.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-09T04:35:25.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-09T04:35:25.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-09T04:35:25.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-09T04:35:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.204+0000 7f03c0b83700 1 Processor -- start 2023-06-09T04:35:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.204+0000 7f03c0b83700 1 -- start start 2023-06-09T04:35:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.206+0000 7f03c0b83700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03bc0cef40 0x7f03bc070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.206+0000 7f03c0b83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.206+0000 7f03c0b83700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc071210 con 0x7f03bc158660 2023-06-09T04:35:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.206+0000 7f03c0b83700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03bc0ae180 con 0x7f03bc0cef40 2023-06-09T04:35:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.206+0000 7f03c0b83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc06aa10 con 0x7f03bc15c680 2023-06-09T04:35:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.207+0000 7f03b9d9b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f03bc158660 0x7f03bc0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58692/0 (socket says 172.21.15.92:58692) 2023-06-09T04:35:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.207+0000 7f03b9d9b700 1 -- 172.21.15.92:0/2396789360 learned_addr learned my addr 172.21.15.92:0/2396789360 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.207+0000 7f03ba59c700 1 --2- 172.21.15.92:0/2396789360 >> v2:172.21.15.123:3300/0 conn(0x7f03bc0cef40 0x7f03bc070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.207+0000 7f03bad9d700 1 --2- 172.21.15.92:0/2396789360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.208+0000 7f03b959a700 1 -- 172.21.15.92:0/2396789360 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082221764 0 0) 0x7f03ac001160 con 0x7f03bc158660 2023-06-09T04:35:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.208+0000 7f03bad9d700 1 -- 172.21.15.92:0/2396789360 >> v2:172.21.15.123:3300/0 conn(0x7f03bc0cef40 msgr2=0x7f03bc070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.208+0000 7f03bad9d700 1 --2- 172.21.15.92:0/2396789360 >> v2:172.21.15.123:3300/0 conn(0x7f03bc0cef40 0x7f03bc070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.208+0000 7f03bad9d700 1 -- 172.21.15.92:0/2396789360 >> v1:172.21.15.123:6789/0 conn(0x7f03bc158660 legacy=0x7f03bc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.208+0000 7f03bad9d700 1 -- 172.21.15.92:0/2396789360 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc06a170 con 0x7f03bc15c680 2023-06-09T04:35:25.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.208+0000 7f03bad9d700 1 --2- 172.21.15.92:0/2396789360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc15cb00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009d90 tx=0x7f03b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f376ca842a0b86a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.209+0000 7f03b959a700 1 -- 172.21.15.92:0/2396789360 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400e070 con 0x7f03bc15c680 2023-06-09T04:35:25.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.209+0000 7f03b959a700 1 -- 172.21.15.92:0/2396789360 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b400bb20 con 0x7f03bc15c680 2023-06-09T04:35:25.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.209+0000 7f03b959a700 1 -- 172.21.15.92:0/2396789360 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4012430 con 0x7f03bc15c680 2023-06-09T04:35:25.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.210+0000 7f03c0b83700 1 -- 172.21.15.92:0/2396789360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 msgr2=0x7f03bc15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.210+0000 7f03c0b83700 1 --2- 172.21.15.92:0/2396789360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc15cb00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009d90 tx=0x7f03b400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.210+0000 7f03c0b83700 1 -- 172.21.15.92:0/2396789360 shutdown_connections 2023-06-09T04:35:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.210+0000 7f03c0b83700 1 --2- 172.21.15.92:0/2396789360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc15cb00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.210+0000 7f03c0b83700 1 --2- 172.21.15.92:0/2396789360 >> v2:172.21.15.123:3300/0 conn(0x7f03bc0cef40 0x7f03bc070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.210+0000 7f03c0b83700 1 -- 172.21.15.92:0/2396789360 >> 172.21.15.92:0/2396789360 conn(0x7f03bc0b0ff0 msgr2=0x7f03bc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.211+0000 7f03c0b83700 1 -- 172.21.15.92:0/2396789360 shutdown_connections 2023-06-09T04:35:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.211+0000 7f03c0b83700 1 -- 172.21.15.92:0/2396789360 wait complete. 2023-06-09T04:35:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.213+0000 7f03c0b83700 1 Processor -- start 2023-06-09T04:35:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.213+0000 7f03c0b83700 1 -- start start 2023-06-09T04:35:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.214+0000 7f03c0b83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03bc158660 0x7f03bc149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.214+0000 7f03c0b83700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.214+0000 7f03c0b83700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0c3ff0 0x7f03bc0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.214+0000 7f03c0b83700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0c7f30 con 0x7f03bc0c3ff0 2023-06-09T04:35:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.215+0000 7f03c0b83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc1495b0 con 0x7f03bc158660 2023-06-09T04:35:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.215+0000 7f03c0b83700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc14b020 con 0x7f03bc15c680 2023-06-09T04:35:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.215+0000 7f03b9d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.215+0000 7f03b9d9b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc0c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46778/0 (socket says 172.21.15.92:46778) 2023-06-09T04:35:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.215+0000 7f03b9d9b700 1 -- 172.21.15.92:0/86703080 learned_addr learned my addr 172.21.15.92:0/86703080 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.216+0000 7f03ba59c700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03bc158660 0x7f03bc149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.216+0000 7f03b9d9b700 1 -- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03bc158660 msgr2=0x7f03bc149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.216+0000 7f03b9d9b700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03bc158660 0x7f03bc149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.216+0000 7f03b9d9b700 1 -- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0c3ff0 msgr2=0x7f03bc0c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.216+0000 7f03b9d9b700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0c3ff0 0x7f03bc0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.216+0000 7f03b9d9b700 1 -- 172.21.15.92:0/86703080 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4009a70 con 0x7f03bc15c680 2023-06-09T04:35:25.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.218+0000 7f03b9d9b700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc0c7070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c460 tx=0x7f03ac00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.219+0000 7f03aaffd700 1 -- 172.21.15.92:0/86703080 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac004c80 con 0x7f03bc15c680 2023-06-09T04:35:25.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.219+0000 7f03aaffd700 1 -- 172.21.15.92:0/86703080 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac004e00 con 0x7f03bc15c680 2023-06-09T04:35:25.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.219+0000 7f03aaffd700 1 -- 172.21.15.92:0/86703080 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac007310 con 0x7f03bc15c680 2023-06-09T04:35:25.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.219+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc0c3530 con 0x7f03bc15c680 2023-06-09T04:35:25.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.219+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc0c37f0 con 0x7f03bc15c680 2023-06-09T04:35:25.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.221+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a00052f0 con 0x7f03bc15c680 2023-06-09T04:35:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.223+0000 7f03aaffd700 1 -- 172.21.15.92:0/86703080 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03ac007470 con 0x7f03bc15c680 2023-06-09T04:35:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.223+0000 7f03aaffd700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f039c06adb0 0x7f039c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.223+0000 7f03aaffd700 1 -- 172.21.15.92:0/86703080 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03ac0895f0 con 0x7f03bc15c680 2023-06-09T04:35:25.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.225+0000 7f03ba59c700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f039c06adb0 0x7f039c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.226+0000 7f03ba59c700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f039c06adb0 0x7f039c06d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03bc15a020 tx=0x7f03b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.229+0000 7f03aaffd700 1 -- 172.21.15.92:0/86703080 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f03ac054680 con 0x7f03bc15c680 2023-06-09T04:35:25.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.254+0000 7f7bc1d31700 1 Processor -- start 2023-06-09T04:35:25.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.254+0000 7f7bc1d31700 1 -- start start 2023-06-09T04:35:25.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.255+0000 7f7bc1d31700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc070cd0 0x7f7bbc15c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.255+0000 7f7bc1d31700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bbc15cbc0 0x7f7bbc0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.255+0000 7f7bc1d31700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc06aa10 con 0x7f7bbc1586f0 2023-06-09T04:35:25.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.255+0000 7f7bc1d31700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bbc0ae180 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.255+0000 7f7bc1d31700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc06a170 con 0x7f7bbc070cd0 2023-06-09T04:35:25.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.256+0000 7f7bbbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bbc15cbc0 0x7f7bbc0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.256+0000 7f7bbbfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bbc15cbc0 0x7f7bbc0cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37876/0 (socket says 172.21.15.92:37876) 2023-06-09T04:35:25.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.256+0000 7f7bbbfff700 1 -- 172.21.15.92:0/658095509 learned_addr learned my addr 172.21.15.92:0/658095509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bbaffd700 1 --2- 172.21.15.92:0/658095509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc070cd0 0x7f7bbc15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bbbfff700 1 -- 172.21.15.92:0/658095509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc070cd0 msgr2=0x7f7bbc15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bbbfff700 1 --2- 172.21.15.92:0/658095509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc070cd0 0x7f7bbc15c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bbbfff700 1 -- 172.21.15.92:0/658095509 >> v1:172.21.15.123:6789/0 conn(0x7f7bbc1586f0 legacy=0x7f7bbc0c8e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T04:35:25.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bbbfff700 1 -- 172.21.15.92:0/658095509 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc06a450 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bbbfff700 1 --2- 172.21.15.92:0/658095509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc0cc590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0088f0 tx=0x7f7bac00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98698fcd131be023 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bba7fc700 1 -- 172.21.15.92:0/658095509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac010070 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bba7fc700 1 -- 172.21.15.92:0/658095509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00da40 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.257+0000 7f7bba7fc700 1 -- 172.21.15.92:0/658095509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00ddd0 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.258+0000 7f7bc1d31700 1 -- 172.21.15.92:0/658095509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 msgr2=0x7f7bbc0cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.258+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/658095509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc0cc590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0088f0 tx=0x7f7bac00d330 comp rx=0 tx=0).stop 2023-06-09T04:35:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.259+0000 7f7bc1d31700 1 -- 172.21.15.92:0/658095509 shutdown_connections 2023-06-09T04:35:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.259+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/658095509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc0cc590 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.259+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/658095509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc070cd0 0x7f7bbc15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.259+0000 7f7bc1d31700 1 -- 172.21.15.92:0/658095509 >> 172.21.15.92:0/658095509 conn(0x7f7bbc0b0ff0 msgr2=0x7f7bbc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.260+0000 7fa9e6cd3700 1 Processor -- start 2023-06-09T04:35:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.260+0000 7fa9e6cd3700 1 -- start start 2023-06-09T04:35:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.261+0000 7f7bc1d31700 1 -- 172.21.15.92:0/658095509 shutdown_connections 2023-06-09T04:35:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.261+0000 7f7bc1d31700 1 -- 172.21.15.92:0/658095509 wait complete. 2023-06-09T04:35:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.261+0000 7fa9e6cd3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa9e01586f0 0x7fa9e0070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.261+0000 7fa9e6cd3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9e00c8e80 0x7fa9e00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.261+0000 7fa9e6cd3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e0071210 con 0x7fa9e015c680 2023-06-09T04:35:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.261+0000 7fa9e6cd3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e00ae180 con 0x7fa9e00c8e80 2023-06-09T04:35:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.261+0000 7fa9e6cd3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e006aa10 con 0x7fa9e01586f0 2023-06-09T04:35:25.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9e00c8e80 0x7fa9e00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9e00c8e80 0x7fa9e00c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37884/0 (socket says 172.21.15.92:37884) 2023-06-09T04:35:25.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 -- 172.21.15.92:0/466630683 learned_addr learned my addr 172.21.15.92:0/466630683 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7f7bc1d31700 1 Processor -- start 2023-06-09T04:35:25.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 -- 172.21.15.92:0/466630683 >> v2:172.21.15.123:3300/0 conn(0x7fa9e01586f0 msgr2=0x7fa9e0070cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T04:35:25.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 --2- 172.21.15.92:0/466630683 >> v2:172.21.15.123:3300/0 conn(0x7fa9e01586f0 0x7fa9e0070cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 -- 172.21.15.92:0/466630683 >> v1:172.21.15.92:6789/0 conn(0x7fa9e015c680 legacy=0x7fa9e00cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 -- 172.21.15.92:0/466630683 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e006a170 con 0x7fa9e00c8e80 2023-06-09T04:35:25.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7f7bc1d31700 1 -- start start 2023-06-09T04:35:25.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.263+0000 7fa9dffff700 1 --2- 172.21.15.92:0/466630683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e00c8e80 0x7fa9e00c92d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00cd260 tx=0x7fa9d000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae11261317b0829 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7fa9df7fe700 1 -- 172.21.15.92:0/466630683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0005c20 con 0x7fa9e00c8e80 2023-06-09T04:35:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bc1d31700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc1586f0 0x7f7bbc15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bc1d31700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bc1d31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bbc0b2980 0x7f7bbc0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bc1d31700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0b3990 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bc1d31700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0b3c70 con 0x7f7bbc0b2980 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bc1d31700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc14d890 con 0x7f7bbc1586f0 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bbaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bbaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc14cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37890/0 (socket says 172.21.15.92:37890) 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7f7bbaffd700 1 -- 172.21.15.92:0/2052163233 learned_addr learned my addr 172.21.15.92:0/2052163233 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7f7bbaffd700 1 -- 172.21.15.92:0/2052163233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc1586f0 msgr2=0x7f7bbc15b500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7f7bbaffd700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc1586f0 0x7f7bbc15b500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7f7bbaffd700 1 -- 172.21.15.92:0/2052163233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bbc0b2980 msgr2=0x7f7bbc0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7f7bbaffd700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bbc0b2980 0x7f7bbc0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7f7bbaffd700 1 -- 172.21.15.92:0/2052163233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0085d0 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7f7bbaffd700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc14cfb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c7a0 tx=0x7f7bb000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7fa9df7fe700 1 -- 172.21.15.92:0/466630683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0005d80 con 0x7fa9e00c8e80 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.264+0000 7fa9df7fe700 1 -- 172.21.15.92:0/466630683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000dbd0 con 0x7fa9e00c8e80 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/466630683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e00c8e80 msgr2=0x7fa9e00c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/466630683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e00c8e80 0x7fa9e00c92d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00cd260 tx=0x7fa9d000be70 comp rx=0 tx=0).stop 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/466630683 shutdown_connections 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/466630683 >> v2:172.21.15.123:3300/0 conn(0x7fa9e01586f0 0x7fa9e0070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/466630683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e00c8e80 0x7fa9e00c92d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.265+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/466630683 >> 172.21.15.92:0/466630683 conn(0x7fa9e00b0ff0 msgr2=0x7fa9e00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.266+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/466630683 shutdown_connections 2023-06-09T04:35:25.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.266+0000 7f7ba3fff700 1 -- 172.21.15.92:0/2052163233 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0004220 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.267+0000 7fddee1d3700 1 Processor -- start 2023-06-09T04:35:25.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.266+0000 7f7ba3fff700 1 -- 172.21.15.92:0/2052163233 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb00043a0 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.266+0000 7f7ba3fff700 1 -- 172.21.15.92:0/2052163233 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00038e0 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.266+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc14db70 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.266+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc14ffc0 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.266+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/466630683 wait complete. 2023-06-09T04:35:25.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.267+0000 7f0387263700 1 Processor -- start 2023-06-09T04:35:25.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.267+0000 7f0387263700 1 -- start start 2023-06-09T04:35:25.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c0052f0 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7fa9e6cd3700 1 Processor -- start 2023-06-09T04:35:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7fa9e6cd3700 1 -- start start 2023-06-09T04:35:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7fddee1d3700 1 -- start start 2023-06-09T04:35:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fddee1d3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdde8070cd0 0x7fdde815c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fddee1d3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde806aa10 con 0x7fdde815cbc0 2023-06-09T04:35:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fddee1d3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde806a170 con 0x7fdde81586f0 2023-06-09T04:35:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fddee1d3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde80ae180 con 0x7fdde8070cd0 2023-06-09T04:35:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fdde7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdde815cbc0 0x7fdde80cc590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36230/0 (socket says 172.21.15.92:36230) 2023-06-09T04:35:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fdde7fff700 1 -- 172.21.15.92:0/1168242524 learned_addr learned my addr 172.21.15.92:0/1168242524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7f7ba3fff700 1 -- 172.21.15.92:0/2052163233 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7bb0003a40 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7f7ba3fff700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7ba806ad30 0x7f7ba806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7f7ba3fff700 1 -- 172.21.15.92:0/2052163233 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7bb0088e10 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7f0387263700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f038006d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7f0387263700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f03800ccd60 0x7f0380069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7f0387263700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038006da80 con 0x7f038006e3b0 2023-06-09T04:35:25.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7f0387263700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0380158840 con 0x7f03800ccd60 2023-06-09T04:35:25.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7f0387263700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038006df40 con 0x7f03800702c0 2023-06-09T04:35:25.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.268+0000 7f0384fff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f038006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7f037ffff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f038006e3b0 0x7f03800c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58724/0 (socket says 172.21.15.92:58724) 2023-06-09T04:35:25.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7f037ffff700 1 -- 172.21.15.92:0/713855344 learned_addr learned my addr 172.21.15.92:0/713855344 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:25.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fa9e6cd3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e00c8e80 0x7fa9e0153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fa9e6cd3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e01586f0 0x7fa9e0159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fa9e6cd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e015c680 0x7fa9e00cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fa9e6cd3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e015a5c0 con 0x7fa9e01586f0 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fa9e6cd3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e015a8a0 con 0x7fa9e015c680 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.269+0000 7fa9e6cd3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e015ab80 con 0x7fa9e00c8e80 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e01586f0 0x7fa9e0159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e01586f0 0x7fa9e0159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37902/0 (socket says 172.21.15.92:37902) 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 -- 172.21.15.92:0/87341353 learned_addr learned my addr 172.21.15.92:0/87341353 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9e5270700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e015c680 0x7fa9e00cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9e4a6f700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e00c8e80 0x7fa9e0153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fada5c40700 1 Processor -- start 2023-06-09T04:35:25.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7f7bbb7fe700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7ba806ad30 0x7f7ba806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 -- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e00c8e80 msgr2=0x7fa9e0153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e00c8e80 0x7fa9e0153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 -- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e015c680 msgr2=0x7fa9e00cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e015c680 0x7fa9e00cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fa9dffff700 1 -- 172.21.15.92:0/87341353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d000b7a0 con 0x7fa9e01586f0 2023-06-09T04:35:25.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7f7bbb7fe700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7ba806ad30 0x7f7ba806d1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc15ad50 tx=0x7f7bb4009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7f037f7fe700 1 -- 172.21.15.92:0/713855344 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431042523 0 0) 0x7f038006da80 con 0x7f038006e3b0 2023-06-09T04:35:25.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7f037f7fe700 1 -- 172.21.15.92:0/713855344 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03680035b0 con 0x7f038006e3b0 2023-06-09T04:35:25.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7f0385800700 1 --2- 172.21.15.92:0/713855344 >> v2:172.21.15.92:3300/0 conn(0x7f03800ccd60 0x7f0380069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7f037f7fe700 1 -- 172.21.15.92:0/713855344 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1648515930 0 0) 0x7f03680035b0 con 0x7f038006e3b0 2023-06-09T04:35:25.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7f0384fff700 1 -- 172.21.15.92:0/713855344 >> v2:172.21.15.92:3300/0 conn(0x7f03800ccd60 msgr2=0x7f0380069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7f0384fff700 1 --2- 172.21.15.92:0/713855344 >> v2:172.21.15.92:3300/0 conn(0x7f03800ccd60 0x7f0380069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7f0384fff700 1 -- 172.21.15.92:0/713855344 >> v1:172.21.15.123:6789/0 conn(0x7f038006e3b0 legacy=0x7f03800c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.270+0000 7fada5c40700 1 -- start start 2023-06-09T04:35:25.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7f0384fff700 1 -- 172.21.15.92:0/713855344 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03800c9590 con 0x7f03800702c0 2023-06-09T04:35:25.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fa9dffff700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e01586f0 0x7fa9e0159b80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9e006fe90 tx=0x7fa9d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7f0385800700 1 --2- 172.21.15.92:0/713855344 >> v2:172.21.15.92:3300/0 conn(0x7f03800ccd60 0x7f0380069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:25.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7f0384fff700 1 --2- 172.21.15.92:0/713855344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f038006d540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03800c9780 tx=0x7f0378014760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ee1417d51b24a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fada5c40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fada0070cd0 0x7fada00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fada5c40700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada006aa10 con 0x7fada00cef40 2023-06-09T04:35:25.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fada5c40700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada006a170 con 0x7fada01586f0 2023-06-09T04:35:25.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fada5c40700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada00ae180 con 0x7fada0070cd0 2023-06-09T04:35:25.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fad9f7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fada01586f0 0x7fada00c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:58742/0 (socket says 172.21.15.92:58742) 2023-06-09T04:35:25.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fad9f7fe700 1 -- 172.21.15.92:0/2636516066 learned_addr learned my addr 172.21.15.92:0/2636516066 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:25.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.271+0000 7fad9ffff700 1 --2- 172.21.15.92:0/2636516066 >> v2:172.21.15.92:3300/0 conn(0x7fada0070cd0 0x7fada00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fdde6ffd700 1 --2- 172.21.15.92:0/1168242524 >> v2:172.21.15.123:3300/0 conn(0x7fdde8070cd0 0x7fdde815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7f037f7fe700 1 -- 172.21.15.92:0/713855344 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370001670 con 0x7f03800702c0 2023-06-09T04:35:25.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7f037f7fe700 1 -- 172.21.15.92:0/713855344 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378003a20 con 0x7f03800702c0 2023-06-09T04:35:25.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7f037f7fe700 1 -- 172.21.15.92:0/713855344 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378003db0 con 0x7f03800702c0 2023-06-09T04:35:25.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038509852 0 0) 0x7fdde806a170 con 0x7fdde81586f0 2023-06-09T04:35:25.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd00034d0 con 0x7fdde81586f0 2023-06-09T04:35:25.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fa84672f700 1 Processor -- start 2023-06-09T04:35:25.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fad9ffff700 1 -- 172.21.15.92:0/2636516066 >> v1:172.21.15.123:6789/0 conn(0x7fada01586f0 legacy=0x7fada00c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fad9ffff700 1 -- 172.21.15.92:0/2636516066 >> v1:172.21.15.92:6789/0 conn(0x7fada00cef40 legacy=0x7fada015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fad9ffff700 1 -- 172.21.15.92:0/2636516066 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada006a450 con 0x7fada0070cd0 2023-06-09T04:35:25.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fa84672f700 1 -- start start 2023-06-09T04:35:25.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.272+0000 7fad9ffff700 1 --2- 172.21.15.92:0/2636516066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 0x7fada00cc590 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fad94009840 tx=0x7fad94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4277d79309f5d445 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 -- 172.21.15.92:0/713855344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 msgr2=0x7f038006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 --2- 172.21.15.92:0/713855344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f038006d540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f03800c9780 tx=0x7f0378014760 comp rx=0 tx=0).stop 2023-06-09T04:35:25.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 -- 172.21.15.92:0/713855344 shutdown_connections 2023-06-09T04:35:25.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 --2- 172.21.15.92:0/713855344 >> v2:172.21.15.92:3300/0 conn(0x7f03800ccd60 0x7f0380069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 --2- 172.21.15.92:0/713855344 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f038006d540 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 -- 172.21.15.92:0/713855344 >> 172.21.15.92:0/713855344 conn(0x7f03800b0ff0 msgr2=0x7f03800ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3072635668 0 0) 0x7fddd00034d0 con 0x7fdde81586f0 2023-06-09T04:35:25.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 >> v2:172.21.15.123:3300/0 conn(0x7fdde8070cd0 msgr2=0x7fdde815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 --2- 172.21.15.92:0/1168242524 >> v2:172.21.15.123:3300/0 conn(0x7fdde8070cd0 0x7fdde815c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 >> v1:172.21.15.92:6789/0 conn(0x7fdde815cbc0 legacy=0x7fdde80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde806a450 con 0x7fdde81586f0 2023-06-09T04:35:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fdddc001670 con 0x7fdde81586f0 2023-06-09T04:35:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 >> v1:172.21.15.123:6789/0 conn(0x7fdde81586f0 legacy=0x7fdde80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd0003af0 0x7fddd0003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddd00088e0 0x7fddd000ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddd000b500 0x7fddd000d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd000e260 con 0x7fddd0003af0 2023-06-09T04:35:25.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd000e3d0 con 0x7fddd00088e0 2023-06-09T04:35:25.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd000e590 con 0x7fddd000b500 2023-06-09T04:35:25.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 -- 172.21.15.92:0/713855344 shutdown_connections 2023-06-09T04:35:25.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde7fff700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddd000b500 0x7fddd000d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fdde77fe700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd0003af0 0x7fddd0003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7f0387263700 1 -- 172.21.15.92:0/713855344 wait complete. 2023-06-09T04:35:25.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde6ffd700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddd00088e0 0x7fddd000ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.273+0000 7fa84672f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa840070b80 0x7fa840158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde7fff700 1 -- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddd00088e0 msgr2=0x7fddd000ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde7fff700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddd00088e0 0x7fddd000ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde7fff700 1 -- 172.21.15.92:0/1168242524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd0003af0 msgr2=0x7fddd0003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde7fff700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd0003af0 0x7fddd0003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde7fff700 1 -- 172.21.15.92:0/1168242524 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde806a450 con 0x7fddd000b500 2023-06-09T04:35:25.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde7fff700 1 -- 172.21.15.92:0/1168242524 reap_dead start 2023-06-09T04:35:25.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa84672f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84006cff0 con 0x7fa84006efc0 2023-06-09T04:35:25.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa84672f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa840159db0 con 0x7fa84006cb80 2023-06-09T04:35:25.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa84672f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa840159230 con 0x7fa840070b80 2023-06-09T04:35:25.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde7fff700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddd000b500 0x7fddd000d9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdde0008bc0 tx=0x7fddd0003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73cae6904cdf517 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde6ffd700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fddd00088e0 0x7fddd000ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00083d0 con 0x7fddd000b500 2023-06-09T04:35:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa83ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa840070b80 0x7fa840158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa83ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa840070b80 0x7fa840158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37926/0 (socket says 172.21.15.92:37926) 2023-06-09T04:35:25.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa83ffff700 1 -- 172.21.15.92:0/3440316176 learned_addr learned my addr 172.21.15.92:0/3440316176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa83ffff700 1 -- 172.21.15.92:0/3440316176 >> v1:172.21.15.123:6789/0 conn(0x7fa84006cb80 legacy=0x7fa840070f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T04:35:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa83ffff700 1 -- 172.21.15.92:0/3440316176 >> v1:172.21.15.92:6789/0 conn(0x7fa84006efc0 legacy=0x7fa84006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa83ffff700 1 -- 172.21.15.92:0/3440316176 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840159510 con 0x7fa840070b80 2023-06-09T04:35:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.274+0000 7fa83ffff700 1 --2- 172.21.15.92:0/3440316176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840070b80 0x7fa840158bc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8300098f0 tx=0x7fa830009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b89f28c2d1063e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.275+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde0001970 con 0x7fddd000b500 2023-06-09T04:35:25.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.275+0000 7fdde67fc700 1 -- 172.21.15.92:0/1168242524 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0001ce0 con 0x7fddd000b500 2023-06-09T04:35:25.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.275+0000 7f0387263700 1 Processor -- start 2023-06-09T04:35:25.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.275+0000 7f0387263700 1 -- start start 2023-06-09T04:35:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.275+0000 7fddee1d3700 1 -- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddd000b500 msgr2=0x7fddd000d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.275+0000 7fddee1d3700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddd000b500 0x7fddd000d9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdde0008bc0 tx=0x7fddd0003780 comp rx=0 tx=0).stop 2023-06-09T04:35:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fddee1d3700 1 -- 172.21.15.92:0/1168242524 shutdown_connections 2023-06-09T04:35:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fddee1d3700 1 --2- 172.21.15.92:0/1168242524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fddd000b500 0x7fddd000d9b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fddee1d3700 1 -- 172.21.15.92:0/1168242524 >> 172.21.15.92:0/1168242524 conn(0x7fdde80b0ff0 msgr2=0x7fdde80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fddee1d3700 1 -- 172.21.15.92:0/1168242524 shutdown_connections 2023-06-09T04:35:25.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fddee1d3700 1 -- 172.21.15.92:0/1168242524 wait complete. 2023-06-09T04:35:25.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7f7ba3fff700 1 -- 172.21.15.92:0/2052163233 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7bb0053fa0 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fa9dd7fa700 1 -- 172.21.15.92:0/87341353 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0009040 con 0x7fa9e01586f0 2023-06-09T04:35:25.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fddee1d3700 1 Processor -- start 2023-06-09T04:35:25.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7f0387263700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f038006e3b0 0x7f03800c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7f0387263700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f03800c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7f0387263700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03800ccd60 0x7f03800a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7f0387263700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03800c5750 con 0x7f038006e3b0 2023-06-09T04:35:25.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7f0387263700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03800b2260 con 0x7f03800ccd60 2023-06-09T04:35:25.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7f0387263700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03800c82a0 con 0x7f03800702c0 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f03800c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f03800c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46846/0 (socket says 172.21.15.92:46846) 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 -- 172.21.15.92:0/4015121077 learned_addr learned my addr 172.21.15.92:0/4015121077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f0384fff700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f038006e3b0 0x7f03800c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 -- 172.21.15.92:0/4015121077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03800ccd60 msgr2=0x7f03800a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03800ccd60 0x7f03800a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 -- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f038006e3b0 msgr2=0x7f03800c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f038006e3b0 0x7f03800c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7f037ffff700 1 -- 172.21.15.92:0/4015121077 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03780140e0 con 0x7f03800702c0 2023-06-09T04:35:25.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7fa9dd7fa700 1 -- 172.21.15.92:0/87341353 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0004450 con 0x7fa9e01586f0 2023-06-09T04:35:25.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7fa9dd7fa700 1 -- 172.21.15.92:0/87341353 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00038c0 con 0x7fa9e01586f0 2023-06-09T04:35:25.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e015ae60 con 0x7fa9e01586f0 2023-06-09T04:35:25.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e015a0c0 con 0x7fa9e01586f0 2023-06-09T04:35:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c00052f0 con 0x7fa9e01586f0 2023-06-09T04:35:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fa83effd700 1 -- 172.21.15.92:0/3440316176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830004230 con 0x7fa840070b80 2023-06-09T04:35:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fa83effd700 1 -- 172.21.15.92:0/3440316176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830004390 con 0x7fa840070b80 2023-06-09T04:35:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fa83effd700 1 -- 172.21.15.92:0/3440316176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000f740 con 0x7fa840070b80 2023-06-09T04:35:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7fa84672f700 1 -- 172.21.15.92:0/3440316176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840070b80 msgr2=0x7fa840158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.277+0000 7fa84672f700 1 --2- 172.21.15.92:0/3440316176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840070b80 0x7fa840158bc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8300098f0 tx=0x7fa830009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fad9e7fc700 1 -- 172.21.15.92:0/2636516066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94011830 con 0x7fada0070cd0 2023-06-09T04:35:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fad9e7fc700 1 -- 172.21.15.92:0/2636516066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad94011990 con 0x7fada0070cd0 2023-06-09T04:35:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.276+0000 7fad9e7fc700 1 -- 172.21.15.92:0/2636516066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94011d00 con 0x7fada0070cd0 2023-06-09T04:35:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fada5c40700 1 -- 172.21.15.92:0/2636516066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 msgr2=0x7fada00cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fada5c40700 1 --2- 172.21.15.92:0/2636516066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 0x7fada00cc590 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fad94009840 tx=0x7fad94009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fada5c40700 1 -- 172.21.15.92:0/2636516066 shutdown_connections 2023-06-09T04:35:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fada5c40700 1 --2- 172.21.15.92:0/2636516066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 0x7fada00cc590 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fada5c40700 1 -- 172.21.15.92:0/2636516066 >> 172.21.15.92:0/2636516066 conn(0x7fada00b0ff0 msgr2=0x7fada00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fddee1d3700 1 -- start start 2023-06-09T04:35:25.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fa84672f700 1 -- 172.21.15.92:0/3440316176 shutdown_connections 2023-06-09T04:35:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fa84672f700 1 --2- 172.21.15.92:0/3440316176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840070b80 0x7fa840158bc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fa84672f700 1 -- 172.21.15.92:0/3440316176 >> 172.21.15.92:0/3440316176 conn(0x7fa8400cf030 msgr2=0x7fa8400cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fada5c40700 1 -- 172.21.15.92:0/2636516066 shutdown_connections 2023-06-09T04:35:25.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.278+0000 7fada5c40700 1 -- 172.21.15.92:0/2636516066 wait complete. 2023-06-09T04:35:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fddee1d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd000b500 0x7fdde8154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fddee1d3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde8156820 0x7fdde80b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fddee1d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde80b2a20 0x7fdde80b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fddee1d3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde815a440 con 0x7fddd000b500 2023-06-09T04:35:25.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fddee1d3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde80b2f30 con 0x7fdde80b2a20 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fddee1d3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8159b10 con 0x7fdde8156820 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fdde77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd000b500 0x7fdde8154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fdde77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd000b500 0x7fdde8154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37942/0 (socket says 172.21.15.92:37942) 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fdde77fe700 1 -- 172.21.15.92:0/2658044007 learned_addr learned my addr 172.21.15.92:0/2658044007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.279+0000 7fdde6ffd700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde8156820 0x7fdde80b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fdde77fe700 1 -- 172.21.15.92:0/2658044007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde8156820 msgr2=0x7fdde80b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fdde77fe700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde8156820 0x7fdde80b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fdde77fe700 1 -- 172.21.15.92:0/2658044007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde80b2a20 msgr2=0x7fdde80b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fdde77fe700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde80b2a20 0x7fdde80b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fdde77fe700 1 -- 172.21.15.92:0/2658044007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8159df0 con 0x7fddd000b500 2023-06-09T04:35:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fada5c40700 1 Processor -- start 2023-06-09T04:35:25.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fdde77fe700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd000b500 0x7fdde8154ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00fae0 tx=0x7fdddc00fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.280+0000 7fada5c40700 1 -- start start 2023-06-09T04:35:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fada5c40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 0x7fada0157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fada5c40700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fada00cef40 0x7fada00c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fada5c40700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada01586f0 0x7fada0153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fada5c40700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada00c3d60 con 0x7fada0070cd0 2023-06-09T04:35:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fada5c40700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada00c2d30 con 0x7fada01586f0 2023-06-09T04:35:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fada5c40700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada00c2fc0 con 0x7fada00cef40 2023-06-09T04:35:25.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7f037ffff700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f03800c7950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f037000c700 tx=0x7f037000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fad9effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fada00cef40 0x7fada00c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fad9ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada01586f0 0x7fada0153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fad9effd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fada00cef40 0x7fada00c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46856/0 (socket says 172.21.15.92:46856) 2023-06-09T04:35:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.281+0000 7fad9effd700 1 -- 172.21.15.92:0/2696156111 learned_addr learned my addr 172.21.15.92:0/2696156111 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad9ffff700 1 -- 172.21.15.92:0/2696156111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fada00cef40 msgr2=0x7fada00c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad9ffff700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fada00cef40 0x7fada00c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad9ffff700 1 -- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 msgr2=0x7fada0157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad9ffff700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 0x7fada0157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad9ffff700 1 -- 172.21.15.92:0/2696156111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940094a0 con 0x7fada01586f0 2023-06-09T04:35:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad9ffff700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada01586f0 0x7fada0153be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad94016000 tx=0x7fad94004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad87fff700 1 -- 172.21.15.92:0/2696156111 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad940036a0 con 0x7fada01586f0 2023-06-09T04:35:25.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fa84672f700 1 -- 172.21.15.92:0/3440316176 shutdown_connections 2023-06-09T04:35:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad87fff700 1 -- 172.21.15.92:0/2696156111 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9401e070 con 0x7fada01586f0 2023-06-09T04:35:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fad87fff700 1 -- 172.21.15.92:0/2696156111 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94003a60 con 0x7fada01586f0 2023-06-09T04:35:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada00c32a0 con 0x7fada01586f0 2023-06-09T04:35:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.282+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada00c3860 con 0x7fada01586f0 2023-06-09T04:35:25.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.283+0000 7fa9dd7fa700 1 -- 172.21.15.92:0/87341353 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa9d0015030 con 0x7fa9e01586f0 2023-06-09T04:35:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.283+0000 7fa9dd7fa700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa9cc06ad60 0x7fa9cc06d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.283+0000 7fa9dd7fa700 1 -- 172.21.15.92:0/87341353 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa9d008a390 con 0x7fa9e01586f0 2023-06-09T04:35:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.283+0000 7fa84672f700 1 -- 172.21.15.92:0/3440316176 wait complete. 2023-06-09T04:35:25.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.283+0000 7fa9e4a6f700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa9cc06ad60 0x7fa9cc06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.284+0000 7fa9e4a6f700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa9cc06ad60 0x7fa9cc06d210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009fd0 tx=0x7fa9d800d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.284+0000 7f037d7fa700 1 -- 172.21.15.92:0/4015121077 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370004c80 con 0x7f03800702c0 2023-06-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.284+0000 7f037d7fa700 1 -- 172.21.15.92:0/4015121077 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0370004e00 con 0x7f03800702c0 2023-06-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.284+0000 7f037d7fa700 1 -- 172.21.15.92:0/4015121077 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370007310 con 0x7f03800702c0 2023-06-09T04:35:25.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.284+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03800b24b0 con 0x7f03800702c0 2023-06-09T04:35:25.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.284+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03800b2a70 con 0x7f03800702c0 2023-06-09T04:35:25.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.285+0000 7fa84672f700 1 Processor -- start 2023-06-09T04:35:25.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.286+0000 7fad87fff700 1 -- 172.21.15.92:0/2696156111 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad94003bc0 con 0x7fada01586f0 2023-06-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.286+0000 7f037d7fa700 1 -- 172.21.15.92:0/4015121077 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0370007470 con 0x7f03800702c0 2023-06-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.286+0000 7fad87fff700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fad8c06ade0 0x7fad8c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.286+0000 7fad87fff700 1 -- 172.21.15.92:0/2696156111 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad9408a3c0 con 0x7fada01586f0 2023-06-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.286+0000 7f037d7fa700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f036c06ad30 0x7f036c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7f037d7fa700 1 -- 172.21.15.92:0/4015121077 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0370089000 con 0x7f03800702c0 2023-06-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fad9f7fe700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fad8c06ade0 0x7fad8c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fddcffff700 1 -- 172.21.15.92:0/2658044007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc001650 con 0x7fddd000b500 2023-06-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fddcffff700 1 -- 172.21.15.92:0/2658044007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc0052e0 con 0x7fddd000b500 2023-06-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fddcffff700 1 -- 172.21.15.92:0/2658044007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc003680 con 0x7fddd000b500 2023-06-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde815a0d0 con 0x7fddd000b500 2023-06-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde80b4420 con 0x7fddd000b500 2023-06-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa9dd7fa700 1 -- 172.21.15.92:0/87341353 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9d00556d0 con 0x7fa9e01586f0 2023-06-09T04:35:25.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fad9f7fe700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fad8c06ade0 0x7fad8c06d290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fada00c40a0 tx=0x7fad90007ad0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03600052f0 con 0x7f03800702c0 2023-06-09T04:35:25.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.286+0000 7fa84672f700 1 -- start start 2023-06-09T04:35:25.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa84672f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84006cb80 0x7fa8400c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa84672f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84006efc0 0x7fa84014e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa84672f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa840070b80 0x7fa84014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa84672f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400c3bc0 con 0x7fa84006cb80 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa84672f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84014e910 con 0x7fa84006efc0 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa84672f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84014eca0 con 0x7fa840070b80 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa844ccc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa840070b80 0x7fa84014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa844ccc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa840070b80 0x7fa84014cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:46866/0 (socket says 172.21.15.92:46866) 2023-06-09T04:35:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa844ccc700 1 -- 172.21.15.92:0/2856212070 learned_addr learned my addr 172.21.15.92:0/2856212070 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa83ffff700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84006cb80 0x7fa8400c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa844ccc700 1 -- 172.21.15.92:0/2856212070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84006efc0 msgr2=0x7fa84014e3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa844ccc700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84006efc0 0x7fa84014e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa844ccc700 1 -- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84006cb80 msgr2=0x7fa8400c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.287+0000 7fa844ccc700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84006cb80 0x7fa8400c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7fa844ccc700 1 -- 172.21.15.92:0/2856212070 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830009550 con 0x7fa840070b80 2023-06-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7fa844ccc700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa840070b80 0x7fa84014cee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa83800cbc0 tx=0x7fa83800ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7fa83cff9700 1 -- 172.21.15.92:0/2856212070 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838003a00 con 0x7fa840070b80 2023-06-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7fa83cff9700 1 -- 172.21.15.92:0/2856212070 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838003b60 con 0x7fa840070b80 2023-06-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7fa83cff9700 1 -- 172.21.15.92:0/2856212070 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83800a840 con 0x7fa840070b80 2023-06-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84014ef80 con 0x7fa840070b80 2023-06-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.288+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8400c3850 con 0x7fa840070b80 2023-06-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.289+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad800052f0 con 0x7fada01586f0 2023-06-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.289+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8400c3d70 con 0x7fa840070b80 2023-06-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.290+0000 7fddcffff700 1 -- 172.21.15.92:0/2658044007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdddc0037e0 con 0x7fddd000b500 2023-06-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.290+0000 7fddcffff700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fddd006c960 0x7fddd006ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.290+0000 7fddcffff700 1 -- 172.21.15.92:0/2658044007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdddc0892c0 con 0x7fddd000b500 2023-06-09T04:35:25.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.290+0000 7fdde6ffd700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fddd006c960 0x7fddd006ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.291+0000 7fdde6ffd700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fddd006c960 0x7fddd006ee10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdde815b670 tx=0x7fddd8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.295+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde815a260 con 0x7fddd000b500 2023-06-09T04:35:25.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.295+0000 7f0384fff700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f036c06ad30 0x7f036c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.295+0000 7f0384fff700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f036c06ad30 0x7f036c06d1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0378000f80 tx=0x7f0378011030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.296+0000 7f037d7fa700 1 -- 172.21.15.92:0/4015121077 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0370054220 con 0x7f03800702c0 2023-06-09T04:35:25.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.298+0000 7fa83cff9700 1 -- 172.21.15.92:0/2856212070 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa83800a9a0 con 0x7fa840070b80 2023-06-09T04:35:25.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.298+0000 7fa83cff9700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa82c06ace0 0x7fa82c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.298+0000 7fa83cff9700 1 -- 172.21.15.92:0/2856212070 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8380abac0 con 0x7fa840070b80 2023-06-09T04:35:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.300+0000 7fa3fca91700 1 Processor -- start 2023-06-09T04:35:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.300+0000 7fa3fca91700 1 -- start start 2023-06-09T04:35:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3fca91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3f815a1d0 0x7fa3f80ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3fca91700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa3f815e800 0x7fa3f80cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3fca91700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f80cbf20 0x7fa3f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3fca91700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f806b200 con 0x7fa3f815a1d0 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3fca91700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f80b1e90 con 0x7fa3f815e800 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3fca91700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80cd480 con 0x7fa3f80cbf20 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3f659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3f815a1d0 0x7fa3f80ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3f659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3f815a1d0 0x7fa3f80ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37964/0 (socket says 172.21.15.92:37964) 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3f659c700 1 -- 172.21.15.92:0/2404988235 learned_addr learned my addr 172.21.15.92:0/2404988235 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3f5d9b700 1 --2- 172.21.15.92:0/2404988235 >> v2:172.21.15.123:3300/0 conn(0x7fa3f815e800 0x7fa3f80cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.301+0000 7fa3f6d9d700 1 --2- 172.21.15.92:0/2404988235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f80cbf20 0x7fa3f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.302+0000 7fa3f659c700 1 -- 172.21.15.92:0/2404988235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f80cbf20 msgr2=0x7fa3f8069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.302+0000 7fa3f659c700 1 --2- 172.21.15.92:0/2404988235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f80cbf20 0x7fa3f8069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.302+0000 7fa3f659c700 1 -- 172.21.15.92:0/2404988235 >> v2:172.21.15.123:3300/0 conn(0x7fa3f815e800 msgr2=0x7fa3f80cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.302+0000 7fa3f659c700 1 --2- 172.21.15.92:0/2404988235 >> v2:172.21.15.123:3300/0 conn(0x7fa3f815e800 0x7fa3f80cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.302+0000 7fa3f659c700 1 -- 172.21.15.92:0/2404988235 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80cc3c0 con 0x7fa3f815a1d0 2023-06-09T04:35:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.302+0000 7fa3f659c700 1 --2- 172.21.15.92:0/2404988235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f815a1d0 0x7fa3f80ccf40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80b4e30 tx=0x7fa3e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c292a012464ed346 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.302+0000 7fa83cff9700 1 -- 172.21.15.92:0/2856212070 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa838076c50 con 0x7fa840070b80 2023-06-09T04:35:25.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.304+0000 7fa83ffff700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa82c06ace0 0x7fa82c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.304+0000 7fddcffff700 1 -- 172.21.15.92:0/2658044007 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdddc054450 con 0x7fddd000b500 2023-06-09T04:35:25.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.305+0000 7fa83ffff700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa82c06ace0 0x7fa82c06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa830000c00 tx=0x7fa830008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.304+0000 7fad87fff700 1 -- 172.21.15.92:0/2696156111 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fad94020080 con 0x7fada01586f0 2023-06-09T04:35:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.306+0000 7fa3f559a700 1 -- 172.21.15.92:0/2404988235 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8011830 con 0x7fa3f815a1d0 2023-06-09T04:35:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.306+0000 7fa3f559a700 1 -- 172.21.15.92:0/2404988235 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8011990 con 0x7fa3f815a1d0 2023-06-09T04:35:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.306+0000 7fa3f559a700 1 -- 172.21.15.92:0/2404988235 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8011d20 con 0x7fa3f815a1d0 2023-06-09T04:35:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.307+0000 7fa3fca91700 1 -- 172.21.15.92:0/2404988235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f815a1d0 msgr2=0x7fa3f80ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.307+0000 7fa3fca91700 1 --2- 172.21.15.92:0/2404988235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f815a1d0 0x7fa3f80ccf40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80b4e30 tx=0x7fa3e8009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.307+0000 7fa3fca91700 1 -- 172.21.15.92:0/2404988235 shutdown_connections 2023-06-09T04:35:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.307+0000 7fa3fca91700 1 --2- 172.21.15.92:0/2404988235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f80cbf20 0x7fa3f8069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.307+0000 7fa3fca91700 1 --2- 172.21.15.92:0/2404988235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f815a1d0 0x7fa3f80ccf40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.307+0000 7fa3fca91700 1 --2- 172.21.15.92:0/2404988235 >> v2:172.21.15.123:3300/0 conn(0x7fa3f815e800 0x7fa3f80cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.307+0000 7fa3fca91700 1 -- 172.21.15.92:0/2404988235 >> 172.21.15.92:0/2404988235 conn(0x7fa3f80b1800 msgr2=0x7fa3f806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.309+0000 7fa3fca91700 1 -- 172.21.15.92:0/2404988235 shutdown_connections 2023-06-09T04:35:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.323+0000 7fa3fca91700 1 -- 172.21.15.92:0/2404988235 wait complete. 2023-06-09T04:35:25.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.325+0000 7fa3fca91700 1 Processor -- start 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.325+0000 7fa3fca91700 1 -- start start 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3fca91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f80cbf20 0x7fa3f8154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3fca91700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f815a1d0 0x7fa3f80c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3fca91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f815e800 0x7fa3f80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3fca91700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81f7590 con 0x7fa3f80cbf20 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3fca91700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80c3630 con 0x7fa3f815e800 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3fca91700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80c2870 con 0x7fa3f815a1d0 2023-06-09T04:35:25.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3f659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f80cbf20 0x7fa3f8154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3f659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f80cbf20 0x7fa3f8154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37974/0 (socket says 172.21.15.92:37974) 2023-06-09T04:35:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.327+0000 7fa3f659c700 1 -- 172.21.15.92:0/4113769765 learned_addr learned my addr 172.21.15.92:0/4113769765 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.328+0000 7fa3f659c700 1 -- 172.21.15.92:0/4113769765 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f815a1d0 msgr2=0x7fa3f80c3860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T04:35:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.328+0000 7fa3f659c700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f815a1d0 0x7fa3f80c3860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.328+0000 7fa3f659c700 1 -- 172.21.15.92:0/4113769765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f815e800 msgr2=0x7fa3f80b3990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T04:35:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.328+0000 7fa3f659c700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f815e800 0x7fa3f80b3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.328+0000 7fa3f659c700 1 -- 172.21.15.92:0/4113769765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8009550 con 0x7fa3f80cbf20 2023-06-09T04:35:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.328+0000 7fa3f659c700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f80cbf20 0x7fa3f8154530 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000c00 tx=0x7fa3e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.332+0000 7fa3e6ffd700 1 -- 172.21.15.92:0/4113769765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8004300 con 0x7fa3f80cbf20 2023-06-09T04:35:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.332+0000 7fa3e6ffd700 1 -- 172.21.15.92:0/4113769765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e801f070 con 0x7fa3f80cbf20 2023-06-09T04:35:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.332+0000 7fa3e6ffd700 1 -- 172.21.15.92:0/4113769765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80036a0 con 0x7fa3f80cbf20 2023-06-09T04:35:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.332+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f80c2b50 con 0x7fa3f80cbf20 2023-06-09T04:35:25.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.332+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f80c3110 con 0x7fa3f80cbf20 2023-06-09T04:35:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.336+0000 7fa3e6ffd700 1 -- 172.21.15.92:0/4113769765 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3e8021030 con 0x7fa3f80cbf20 2023-06-09T04:35:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.337+0000 7fa3e6ffd700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3e006af20 0x7fa3e006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.337+0000 7fa3e6ffd700 1 -- 172.21.15.92:0/4113769765 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3e808a3d0 con 0x7fa3f80cbf20 2023-06-09T04:35:25.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.338+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc0052f0 con 0x7fa3f80cbf20 2023-06-09T04:35:25.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.341+0000 7fa3f5d9b700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3e006af20 0x7fa3e006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.341+0000 7fa3f5d9b700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3e006af20 0x7fa3e006d3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3f815db80 tx=0x7fa3ec009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.348+0000 7fa3e6ffd700 1 -- 172.21.15.92:0/4113769765 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3e80554f0 con 0x7fa3f80cbf20 2023-06-09T04:35:25.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.601+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03a0005870 con 0x7f03bc15c680 2023-06-09T04:35:25.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.601+0000 7f03aaffd700 1 -- 172.21.15.92:0/86703080 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f03ac007720 con 0x7f03bc15c680 2023-06-09T04:35:25.604 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:25.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f039c06adb0 msgr2=0x7f039c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f039c06adb0 0x7f039c06d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03bc15a020 tx=0x7f03b0008040 comp rx=0 tx=0).stop 2023-06-09T04:35:25.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 msgr2=0x7f03bc0c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc0c7070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c460 tx=0x7f03ac00c820 comp rx=0 tx=0).stop 2023-06-09T04:35:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 shutdown_connections 2023-06-09T04:35:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f039c06adb0 0x7f039c06d260 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03bc15a020 tx=0x7f03b0008040 comp rx=0 tx=0).stop 2023-06-09T04:35:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f03bc0c3ff0 0x7f03bc0c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03bc158660 0x7f03bc149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 --2- 172.21.15.92:0/86703080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03bc15c680 0x7f03bc0c7070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 >> 172.21.15.92:0/86703080 conn(0x7f03bc0b0ff0 msgr2=0x7f03bc15a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 shutdown_connections 2023-06-09T04:35:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.605+0000 7f03c0b83700 1 -- 172.21.15.92:0/86703080 wait complete. 2023-06-09T04:35:25.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-06-09T04:35:25.625 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.675+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7b9c005870 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.675+0000 7f7ba3fff700 1 -- 172.21.15.92:0/2052163233 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7bb001e090 con 0x7f7bbc15cbc0 2023-06-09T04:35:25.677 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7ba806ad30 msgr2=0x7f7ba806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7ba806ad30 0x7f7ba806d1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc15ad50 tx=0x7f7bb4009250 comp rx=0 tx=0).stop 2023-06-09T04:35:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 msgr2=0x7f7bbc14cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc14cfb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000c7a0 tx=0x7f7bb000cb60 comp rx=0 tx=0).stop 2023-06-09T04:35:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 shutdown_connections 2023-06-09T04:35:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7ba806ad30 0x7f7ba806d1e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bbc0b2980 0x7f7bbc0b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bbc1586f0 0x7f7bbc15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 --2- 172.21.15.92:0/2052163233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bbc15cbc0 0x7f7bbc14cfb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.680+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 >> 172.21.15.92:0/2052163233 conn(0x7f7bbc0b0ff0 msgr2=0x7f7bbc1538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.681+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 shutdown_connections 2023-06-09T04:35:25.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.681+0000 7f7bc1d31700 1 -- 172.21.15.92:0/2052163233 wait complete. 2023-06-09T04:35:25.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.693+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa3dc0055a0 con 0x7fa3f80cbf20 2023-06-09T04:35:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.695+0000 7fa3e6ffd700 1 -- 172.21.15.92:0/4113769765 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3e8015030 con 0x7fa3f80cbf20 2023-06-09T04:35:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.695+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fad800055a0 con 0x7fada01586f0 2023-06-09T04:35:25.696 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:25.696 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-06-09T04:35:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.696+0000 7fad87fff700 1 -- 172.21.15.92:0/2696156111 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fad94015030 con 0x7fada01586f0 2023-06-09T04:35:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.698+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fad8c06ade0 msgr2=0x7fad8c06d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.698+0000 7fada5c40700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fad8c06ade0 0x7fad8c06d290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fada00c40a0 tx=0x7fad90007ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.698+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada01586f0 msgr2=0x7fada0153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.698+0000 7fada5c40700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada01586f0 0x7fada0153be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad94016000 tx=0x7fad94004750 comp rx=0 tx=0).stop 2023-06-09T04:35:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 shutdown_connections 2023-06-09T04:35:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fada5c40700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fad8c06ade0 0x7fad8c06d290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fada5c40700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fada01586f0 0x7fada0153be0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fada5c40700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fada0070cd0 0x7fada0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fada5c40700 1 --2- 172.21.15.92:0/2696156111 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fada00cef40 0x7fada00c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 >> 172.21.15.92:0/2696156111 conn(0x7fada00b0ff0 msgr2=0x7fada00af160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-06-09T04:35:25.702 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3e006af20 msgr2=0x7fa3e006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fa3fca91700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3e006af20 0x7fa3e006d3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3f815db80 tx=0x7fa3ec009210 comp rx=0 tx=0).stop 2023-06-09T04:35:25.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f80cbf20 msgr2=0x7fa3f8154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.699+0000 7fa3fca91700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f80cbf20 0x7fa3f8154530 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8000c00 tx=0x7fa3e8004750 comp rx=0 tx=0).stop 2023-06-09T04:35:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 shutdown_connections 2023-06-09T04:35:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3e006af20 0x7fa3e006d3d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f815e800 0x7fa3f80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3f80cbf20 0x7fa3f8154530 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 --2- 172.21.15.92:0/4113769765 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3f815a1d0 0x7fa3f80c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 >> 172.21.15.92:0/4113769765 conn(0x7fa3f80b1800 msgr2=0x7fa3f80cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 shutdown_connections 2023-06-09T04:35:25.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.700+0000 7fa3fca91700 1 -- 172.21.15.92:0/4113769765 wait complete. 2023-06-09T04:35:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.705+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa9c0005870 con 0x7fa9e01586f0 2023-06-09T04:35:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.706+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 shutdown_connections 2023-06-09T04:35:25.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.706+0000 7fa9dd7fa700 1 -- 172.21.15.92:0/87341353 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa9d0058cf0 con 0x7fa9e01586f0 2023-06-09T04:35:25.707 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:25.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.706+0000 7fada5c40700 1 -- 172.21.15.92:0/2696156111 wait complete. 2023-06-09T04:35:25.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.707+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa8400c3d30 con 0x7fa840070b80 2023-06-09T04:35:25.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.707+0000 7fa83cff9700 1 -- 172.21.15.92:0/2856212070 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa83807a270 con 0x7fa840070b80 2023-06-09T04:35:25.708 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:25.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.708+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdde80740e0 con 0x7fddd000b500 2023-06-09T04:35:25.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.708+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0360005870 con 0x7f03800702c0 2023-06-09T04:35:25.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.708+0000 7fddcffff700 1 -- 172.21.15.92:0/2658044007 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdddc057a70 con 0x7fddd000b500 2023-06-09T04:35:25.710 INFO:teuthology.orchestra.run.smithi092.stdout:55834574857 2023-06-09T04:35:25.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.709+0000 7f037d7fa700 1 -- 172.21.15.92:0/4015121077 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0370057840 con 0x7f03800702c0 2023-06-09T04:35:25.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.709+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa9cc06ad60 msgr2=0x7fa9cc06d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.709+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa9cc06ad60 0x7fa9cc06d210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009fd0 tx=0x7fa9d800d040 comp rx=0 tx=0).stop 2023-06-09T04:35:25.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.709+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e01586f0 msgr2=0x7fa9e0159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.709+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e01586f0 0x7fa9e0159b80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa9e006fe90 tx=0x7fa9d0004750 comp rx=0 tx=0).stop 2023-06-09T04:35:25.711 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-06-09T04:35:25.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.710+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 shutdown_connections 2023-06-09T04:35:25.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.710+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa9cc06ad60 0x7fa9cc06d210 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009fd0 tx=0x7fa9d800d040 comp rx=0 tx=0).stop 2023-06-09T04:35:25.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.710+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e015c680 0x7fa9e00cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.710+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e00c8e80 0x7fa9e0153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.710+0000 7fa9e6cd3700 1 --2- 172.21.15.92:0/87341353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e01586f0 0x7fa9e0159b80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.710+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 >> 172.21.15.92:0/87341353 conn(0x7fa9e00b0ff0 msgr2=0x7fa9e01563b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa82c06ace0 msgr2=0x7fa82c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa82c06ace0 0x7fa82c06d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa830000c00 tx=0x7fa830008030 comp rx=0 tx=0).stop 2023-06-09T04:35:25.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa840070b80 msgr2=0x7fa84014cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa840070b80 0x7fa84014cee0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa83800cbc0 tx=0x7fa83800ced0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 shutdown_connections 2023-06-09T04:35:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa82c06ace0 0x7fa82c06d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa840070b80 0x7fa84014cee0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84006cb80 0x7fa8400c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 --2- 172.21.15.92:0/2856212070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa84006efc0 0x7fa84014e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.711+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 >> 172.21.15.92:0/2856212070 conn(0x7fa8400cf030 msgr2=0x7fa84006e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 shutdown_connections 2023-06-09T04:35:25.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fa84672f700 1 -- 172.21.15.92:0/2856212070 wait complete. 2023-06-09T04:35:25.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f036c06ad30 msgr2=0x7f036c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7f0387263700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f036c06ad30 0x7f036c06d1e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0378000f80 tx=0x7f0378011030 comp rx=0 tx=0).stop 2023-06-09T04:35:25.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 msgr2=0x7f03800c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7f0387263700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f03800c7950 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f037000c700 tx=0x7f037000ca10 comp rx=0 tx=0).stop 2023-06-09T04:35:25.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fddd006c960 msgr2=0x7fddd006ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fddd006c960 0x7fddd006ee10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdde815b670 tx=0x7fddd8008040 comp rx=0 tx=0).stop 2023-06-09T04:35:25.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd000b500 msgr2=0x7fdde8154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd000b500 0x7fdde8154ab0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00fae0 tx=0x7fdddc00fea0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 shutdown_connections 2023-06-09T04:35:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fddd006c960 0x7fddd006ee10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde80b2a20 0x7fdde80b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddd000b500 0x7fdde8154ab0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 --2- 172.21.15.92:0/2658044007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde8156820 0x7fdde80b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 >> 172.21.15.92:0/2658044007 conn(0x7fdde80b0ff0 msgr2=0x7fdde80ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 shutdown_connections 2023-06-09T04:35:25.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fddee1d3700 1 -- 172.21.15.92:0/2658044007 wait complete. 2023-06-09T04:35:25.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 shutdown_connections 2023-06-09T04:35:25.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.712+0000 7fa9e6cd3700 1 -- 172.21.15.92:0/87341353 wait complete. 2023-06-09T04:35:25.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.714+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 shutdown_connections 2023-06-09T04:35:25.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.714+0000 7f0387263700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f036c06ad30 0x7f036c06d1e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.714+0000 7f0387263700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03800ccd60 0x7f03800a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.714+0000 7f0387263700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f038006e3b0 0x7f03800c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.714+0000 7f0387263700 1 --2- 172.21.15.92:0/4015121077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03800702c0 0x7f03800c7950 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.714+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 >> 172.21.15.92:0/4015121077 conn(0x7f03800b0ff0 msgr2=0x7f038006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.715+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 shutdown_connections 2023-06-09T04:35:25.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.715+0000 7f0387263700 1 -- 172.21.15.92:0/4015121077 wait complete. 2023-06-09T04:35:25.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-06-09T04:35:25.721 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-06-09T04:35:25.731 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-06-09T04:35:25.732 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-06-09T04:35:25.733 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-06-09T04:35:25.733 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-06-09T04:35:25.735 DEBUG:teuthology.parallel:result is None 2023-06-09T04:35:25.735 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-09T04:35:25.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T04:35:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.895+0000 7fe64b8fa700 1 Processor -- start 2023-06-09T04:35:25.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.896+0000 7fe64b8fa700 1 -- start start 2023-06-09T04:35:25.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.896+0000 7fe64b8fa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 0x7fe6440ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.896+0000 7fe64b8fa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6440ce730 con 0x7fe6440ac400 2023-06-09T04:35:25.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.896+0000 7fe64b8fa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64406aaf0 con 0x7fe6440cf010 2023-06-09T04:35:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.896+0000 7fe64b8fa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64415cba0 con 0x7fe64415c7d0 2023-06-09T04:35:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.896+0000 7fe648e95700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe6440ac400 0x7fe6440ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36312/0 (socket says 172.21.15.92:36312) 2023-06-09T04:35:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe648e95700 1 -- 172.21.15.92:0/2111213940 learned_addr learned my addr 172.21.15.92:0/2111213940 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649696700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 0x7fe6440ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015044282 0 0) 0x7fe640001110 con 0x7fe6440ac400 2023-06-09T04:35:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6300035b0 con 0x7fe6440ac400 2023-06-09T04:35:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633952416 0 0) 0x7fe64406aaf0 con 0x7fe6440cf010 2023-06-09T04:35:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe640001110 con 0x7fe6440cf010 2023-06-09T04:35:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4212977972 0 0) 0x7fe6300035b0 con 0x7fe6440ac400 2023-06-09T04:35:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 msgr2=0x7fe6440ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 0x7fe6440ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 >> v1:172.21.15.123:6789/0 conn(0x7fe6440cf010 legacy=0x7fe64406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe644069ee0 con 0x7fe6440ac400 2023-06-09T04:35:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fe640001800 con 0x7fe6440ac400 2023-06-09T04:35:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649696700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 0x7fe6440ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 >> v1:172.21.15.92:6789/0 conn(0x7fe6440ac400 legacy=0x7fe6440ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 0x7fe630003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630004150 0x7fe630004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe630005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64415cba0 con 0x7fe630004d90 2023-06-09T04:35:25.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63000bba0 con 0x7fe630004150 2023-06-09T04:35:25.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63000bd60 con 0x7fe64415c7d0 2023-06-09T04:35:25.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649e97700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe630005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649696700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 0x7fe630003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe648e95700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630004150 0x7fe630004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649e97700 1 -- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 msgr2=0x7fe630003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649e97700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe64415c7d0 0x7fe630003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649e97700 1 -- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630004150 msgr2=0x7fe630004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.897+0000 7fe649e97700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe630004150 0x7fe630004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe649e97700 1 -- 172.21.15.92:0/2111213940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe634009430 con 0x7fe630004d90 2023-06-09T04:35:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe649e97700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe630005200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c490 tx=0x7fe630005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd831b671eef5620 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c003a00 con 0x7fe630004d90 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c003b60 con 0x7fe630004d90 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe63bfff700 1 -- 172.21.15.92:0/2111213940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c0050d0 con 0x7fe630004d90 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe64b8fa700 1 -- 172.21.15.92:0/2111213940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 msgr2=0x7fe630005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe64b8fa700 1 --2- 172.21.15.92:0/2111213940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe630005200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c490 tx=0x7fe630005880 comp rx=0 tx=0).stop 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe649e97700 1 -- 172.21.15.92:0/2111213940 reap_dead start 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe64b8fa700 1 -- 172.21.15.92:0/2111213940 shutdown_connections 2023-06-09T04:35:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe64b8fa700 1 -- 172.21.15.92:0/2111213940 >> 172.21.15.92:0/2111213940 conn(0x7fe644098c10 msgr2=0x7fe6440cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:25.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe64b8fa700 1 -- 172.21.15.92:0/2111213940 shutdown_connections 2023-06-09T04:35:25.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.898+0000 7fe64b8fa700 1 -- 172.21.15.92:0/2111213940 wait complete. 2023-06-09T04:35:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 Processor -- start 2023-06-09T04:35:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 -- start start 2023-06-09T04:35:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe6440c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6440c8a60 0x7fe6440cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6440c42d0 0x7fe64414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6440c4c40 con 0x7fe630004d90 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6440c4f20 con 0x7fe6440c8a60 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe64b8fa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6440c5200 con 0x7fe6440c42d0 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe649696700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe6440c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe649696700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe6440c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37990/0 (socket says 172.21.15.92:37990) 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.900+0000 7fe649696700 1 -- 172.21.15.92:0/1158399020 learned_addr learned my addr 172.21.15.92:0/1158399020 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe649e97700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6440c42d0 0x7fe64414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe648e95700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6440c8a60 0x7fe6440cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe649696700 1 -- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6440c42d0 msgr2=0x7fe64414d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe649696700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6440c42d0 0x7fe64414d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe649696700 1 -- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6440c8a60 msgr2=0x7fe6440cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe649696700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6440c8a60 0x7fe6440cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe649696700 1 -- 172.21.15.92:0/1158399020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6440c54e0 con 0x7fe630004d90 2023-06-09T04:35:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe649696700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe6440c5ce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe634011cd0 tx=0x7fe63400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe639ffb700 1 -- 172.21.15.92:0/1158399020 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63400fb30 con 0x7fe630004d90 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe639ffb700 1 -- 172.21.15.92:0/1158399020 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63400fc90 con 0x7fe630004d90 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe639ffb700 1 -- 172.21.15.92:0/1158399020 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634014440 con 0x7fe630004d90 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6440c57c0 con 0x7fe630004d90 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.901+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6440cce40 con 0x7fe630004d90 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.902+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6440740e0 con 0x7fe630004d90 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.902+0000 7fe639ffb700 1 -- 172.21.15.92:0/1158399020 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6340145a0 con 0x7fe630004d90 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.903+0000 7fe639ffb700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe62806ace0 0x7fe62806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:25.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.903+0000 7fe639ffb700 1 -- 172.21.15.92:0/1158399020 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe634089e10 con 0x7fe630004d90 2023-06-09T04:35:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.903+0000 7fe648e95700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe62806ace0 0x7fe62806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.903+0000 7fe648e95700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe62806ace0 0x7fe62806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe6440c8e30 tx=0x7fe640009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:25.907+0000 7fe639ffb700 1 -- 172.21.15.92:0/1158399020 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe63408d030 con 0x7fe630004d90 2023-06-09T04:35:26.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.093+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 --> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6440c5090 con 0x7fe62806ace0 2023-06-09T04:35:26.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.100+0000 7fe639ffb700 1 -- 172.21.15.92:0/1158399020 <== mgr.4104 v2:172.21.15.92:6832/3112104013 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286634 (secure 0 0 0) 0x7fe6440c5090 con 0x7fe62806ace0 2023-06-09T04:35:26.103 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe62806ace0 msgr2=0x7fe62806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe62806ace0 0x7fe62806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe6440c8e30 tx=0x7fe640009040 comp rx=0 tx=0).stop 2023-06-09T04:35:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 msgr2=0x7fe6440c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe6440c5ce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe634011cd0 tx=0x7fe63400f520 comp rx=0 tx=0).stop 2023-06-09T04:35:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 shutdown_connections 2023-06-09T04:35:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe62806ace0 0x7fe62806d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe6440c42d0 0x7fe64414d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe630004d90 0x7fe6440c5ce0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.103+0000 7fe64b8fa700 1 --2- 172.21.15.92:0/1158399020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6440c8a60 0x7fe6440cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.104+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 >> 172.21.15.92:0/1158399020 conn(0x7fe644098c10 msgr2=0x7fe6440ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.104+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 shutdown_connections 2023-06-09T04:35:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.104+0000 7fe64b8fa700 1 -- 172.21.15.92:0/1158399020 wait complete. 2023-06-09T04:35:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-06-09T04:35:26.116 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-06-09T04:35:25.845950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218820,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755900,"statfs":{"total":773094113280,"available":772870041600,"internally_reserved":0,"allocated":3280896,"data_stored":1938951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.431818"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.956059+0000","last_change":"2023-06-09T04:35:17.427755+0000","last_active":"2023-06-09T04:35:18.956059+0000","last_peered":"2023-06-09T04:35:18.956059+0000","last_clean":"2023-06-09T04:35:18.956059+0000","last_became_active":"2023-06-09T04:35:17.427498+0000","last_became_peered":"2023-06-09T04:35:17.427498+0000","last_unstale":"2023-06-09T04:35:18.956059+0000","last_undegraded":"2023-06-09T04:35:18.956059+0000","last_fullsized":"2023-06-09T04:35:18.956059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:05.638304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423895+0000","last_change":"2023-06-09T04:35:17.438248+0000","last_active":"2023-06-09T04:35:18.423895+0000","last_peered":"2023-06-09T04:35:18.423895+0000","last_clean":"2023-06-09T04:35:18.423895+0000","last_became_active":"2023-06-09T04:35:17.437959+0000","last_became_peered":"2023-06-09T04:35:17.437959+0000","last_unstale":"2023-06-09T04:35:18.423895+0000","last_undegraded":"2023-06-09T04:35:18.423895+0000","last_fullsized":"2023-06-09T04:35:18.423895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071287+0000","last_change":"2023-06-09T04:35:17.444189+0000","last_active":"2023-06-09T04:35:19.071287+0000","last_peered":"2023-06-09T04:35:19.071287+0000","last_clean":"2023-06-09T04:35:19.071287+0000","last_became_active":"2023-06-09T04:35:17.443878+0000","last_became_peered":"2023-06-09T04:35:17.443878+0000","last_unstale":"2023-06-09T04:35:19.071287+0000","last_undegraded":"2023-06-09T04:35:19.071287+0000","last_fullsized":"2023-06-09T04:35:19.071287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:01.857396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423293+0000","last_change":"2023-06-09T04:35:17.439643+0000","last_active":"2023-06-09T04:35:18.423293+0000","last_peered":"2023-06-09T04:35:18.423293+0000","last_clean":"2023-06-09T04:35:18.423293+0000","last_became_active":"2023-06-09T04:35:17.439270+0000","last_became_peered":"2023-06-09T04:35:17.439270+0000","last_unstale":"2023-06-09T04:35:18.423293+0000","last_undegraded":"2023-06-09T04:35:18.423293+0000","last_fullsized":"2023-06-09T04:35:18.423293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:05.638304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353770+0000","last_change":"2023-06-09T04:35:17.440592+0000","last_active":"2023-06-09T04:35:20.353770+0000","last_peered":"2023-06-09T04:35:20.353770+0000","last_clean":"2023-06-09T04:35:20.353770+0000","last_became_active":"2023-06-09T04:35:17.440212+0000","last_became_peered":"2023-06-09T04:35:17.440212+0000","last_unstale":"2023-06-09T04:35:20.353770+0000","last_undegraded":"2023-06-09T04:35:20.353770+0000","last_fullsized":"2023-06-09T04:35:20.353770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:35:03.891489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070853+0000","last_change":"2023-06-09T04:35:17.439583+0000","last_active":"2023-06-09T04:35:19.070853+0000","last_peered":"2023-06-09T04:35:19.070853+0000","last_clean":"2023-06-09T04:35:19.070853+0000","last_became_active":"2023-06-09T04:35:17.439241+0000","last_became_peered":"2023-06-09T04:35:17.439241+0000","last_unstale":"2023-06-09T04:35:19.070853+0000","last_undegraded":"2023-06-09T04:35:19.070853+0000","last_fullsized":"2023-06-09T04:35:19.070853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243375+0000","last_change":"2023-06-09T04:35:17.437095+0000","last_active":"2023-06-09T04:35:20.243375+0000","last_peered":"2023-06-09T04:35:20.243375+0000","last_clean":"2023-06-09T04:35:20.243375+0000","last_became_active":"2023-06-09T04:35:17.436222+0000","last_became_peered":"2023-06-09T04:35:17.436222+0000","last_unstale":"2023-06-09T04:35:20.243375+0000","last_undegraded":"2023-06-09T04:35:20.243375+0000","last_fullsized":"2023-06-09T04:35:20.243375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:01.857396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423746+0000","last_change":"2023-06-09T04:35:17.439627+0000","last_active":"2023-06-09T04:35:18.423746+0000","last_peered":"2023-06-09T04:35:18.423746+0000","last_clean":"2023-06-09T04:35:18.423746+0000","last_became_active":"2023-06-09T04:35:17.439339+0000","last_became_peered":"2023-06-09T04:35:17.439339+0000","last_unstale":"2023-06-09T04:35:18.423746+0000","last_undegraded":"2023-06-09T04:35:18.423746+0000","last_fullsized":"2023-06-09T04:35:18.423746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353891+0000","last_change":"2023-06-09T04:35:17.434172+0000","last_active":"2023-06-09T04:35:20.353891+0000","last_peered":"2023-06-09T04:35:20.353891+0000","last_clean":"2023-06-09T04:35:20.353891+0000","last_became_active":"2023-06-09T04:35:17.433941+0000","last_became_peered":"2023-06-09T04:35:17.433941+0000","last_unstale":"2023-06-09T04:35:20.353891+0000","last_undegraded":"2023-06-09T04:35:20.353891+0000","last_fullsized":"2023-06-09T04:35:20.353891+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423982+0000","last_change":"2023-06-09T04:35:17.441671+0000","last_active":"2023-06-09T04:35:18.423982+0000","last_peered":"2023-06-09T04:35:18.423982+0000","last_clean":"2023-06-09T04:35:18.423982+0000","last_became_active":"2023-06-09T04:35:17.441387+0000","last_became_peered":"2023-06-09T04:35:17.441387+0000","last_unstale":"2023-06-09T04:35:18.423982+0000","last_undegraded":"2023-06-09T04:35:18.423982+0000","last_fullsized":"2023-06-09T04:35:18.423982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071335+0000","last_change":"2023-06-09T04:35:17.438983+0000","last_active":"2023-06-09T04:35:19.071335+0000","last_peered":"2023-06-09T04:35:19.071335+0000","last_clean":"2023-06-09T04:35:19.071335+0000","last_became_active":"2023-06-09T04:35:17.438684+0000","last_became_peered":"2023-06-09T04:35:17.438684+0000","last_unstale":"2023-06-09T04:35:19.071335+0000","last_undegraded":"2023-06-09T04:35:19.071335+0000","last_fullsized":"2023-06-09T04:35:19.071335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:15.755801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955691+0000","last_change":"2023-06-09T04:35:17.433301+0000","last_active":"2023-06-09T04:35:18.955691+0000","last_peered":"2023-06-09T04:35:18.955691+0000","last_clean":"2023-06-09T04:35:18.955691+0000","last_became_active":"2023-06-09T04:35:17.433031+0000","last_became_peered":"2023-06-09T04:35:17.433031+0000","last_unstale":"2023-06-09T04:35:18.955691+0000","last_undegraded":"2023-06-09T04:35:18.955691+0000","last_fullsized":"2023-06-09T04:35:18.955691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955577+0000","last_change":"2023-06-09T04:35:17.432672+0000","last_active":"2023-06-09T04:35:18.955577+0000","last_peered":"2023-06-09T04:35:18.955577+0000","last_clean":"2023-06-09T04:35:18.955577+0000","last_became_active":"2023-06-09T04:35:17.432388+0000","last_became_peered":"2023-06-09T04:35:17.432388+0000","last_unstale":"2023-06-09T04:35:18.955577+0000","last_undegraded":"2023-06-09T04:35:18.955577+0000","last_fullsized":"2023-06-09T04:35:18.955577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424293+0000","last_change":"2023-06-09T04:35:17.439222+0000","last_active":"2023-06-09T04:35:18.424293+0000","last_peered":"2023-06-09T04:35:18.424293+0000","last_clean":"2023-06-09T04:35:18.424293+0000","last_became_active":"2023-06-09T04:35:17.438873+0000","last_became_peered":"2023-06-09T04:35:17.438873+0000","last_unstale":"2023-06-09T04:35:18.424293+0000","last_undegraded":"2023-06-09T04:35:18.424293+0000","last_fullsized":"2023-06-09T04:35:18.424293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243743+0000","last_change":"2023-06-09T04:35:17.437077+0000","last_active":"2023-06-09T04:35:20.243743+0000","last_peered":"2023-06-09T04:35:20.243743+0000","last_clean":"2023-06-09T04:35:20.243743+0000","last_became_active":"2023-06-09T04:35:17.436197+0000","last_became_peered":"2023-06-09T04:35:17.436197+0000","last_unstale":"2023-06-09T04:35:20.243743+0000","last_undegraded":"2023-06-09T04:35:20.243743+0000","last_fullsized":"2023-06-09T04:35:20.243743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:53:57.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070817+0000","last_change":"2023-06-09T04:35:17.443763+0000","last_active":"2023-06-09T04:35:19.070817+0000","last_peered":"2023-06-09T04:35:19.070817+0000","last_clean":"2023-06-09T04:35:19.070817+0000","last_became_active":"2023-06-09T04:35:17.442440+0000","last_became_peered":"2023-06-09T04:35:17.442440+0000","last_unstale":"2023-06-09T04:35:19.070817+0000","last_undegraded":"2023-06-09T04:35:19.070817+0000","last_fullsized":"2023-06-09T04:35:19.070817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:55.606991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354004+0000","last_change":"2023-06-09T04:35:17.433713+0000","last_active":"2023-06-09T04:35:20.354004+0000","last_peered":"2023-06-09T04:35:20.354004+0000","last_clean":"2023-06-09T04:35:20.354004+0000","last_became_active":"2023-06-09T04:35:17.433428+0000","last_became_peered":"2023-06-09T04:35:17.433428+0000","last_unstale":"2023-06-09T04:35:20.354004+0000","last_undegraded":"2023-06-09T04:35:20.354004+0000","last_fullsized":"2023-06-09T04:35:20.354004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071439+0000","last_change":"2023-06-09T04:35:17.432392+0000","last_active":"2023-06-09T04:35:19.071439+0000","last_peered":"2023-06-09T04:35:19.071439+0000","last_clean":"2023-06-09T04:35:19.071439+0000","last_became_active":"2023-06-09T04:35:17.432126+0000","last_became_peered":"2023-06-09T04:35:17.432126+0000","last_unstale":"2023-06-09T04:35:19.071439+0000","last_undegraded":"2023-06-09T04:35:19.071439+0000","last_fullsized":"2023-06-09T04:35:19.071439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:58:27.739525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955661+0000","last_change":"2023-06-09T04:35:17.431541+0000","last_active":"2023-06-09T04:35:18.955661+0000","last_peered":"2023-06-09T04:35:18.955661+0000","last_clean":"2023-06-09T04:35:18.955661+0000","last_became_active":"2023-06-09T04:35:17.431276+0000","last_became_peered":"2023-06-09T04:35:17.431276+0000","last_unstale":"2023-06-09T04:35:18.955661+0000","last_undegraded":"2023-06-09T04:35:18.955661+0000","last_fullsized":"2023-06-09T04:35:18.955661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:02:00.587762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071083+0000","last_change":"2023-06-09T04:35:17.445238+0000","last_active":"2023-06-09T04:35:19.071083+0000","last_peered":"2023-06-09T04:35:19.071083+0000","last_clean":"2023-06-09T04:35:19.071083+0000","last_became_active":"2023-06-09T04:35:17.444919+0000","last_became_peered":"2023-06-09T04:35:17.444919+0000","last_unstale":"2023-06-09T04:35:19.071083+0000","last_undegraded":"2023-06-09T04:35:19.071083+0000","last_fullsized":"2023-06-09T04:35:19.071083+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:04.391938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243586+0000","last_change":"2023-06-09T04:35:17.441967+0000","last_active":"2023-06-09T04:35:20.243586+0000","last_peered":"2023-06-09T04:35:20.243586+0000","last_clean":"2023-06-09T04:35:20.243586+0000","last_became_active":"2023-06-09T04:35:17.441745+0000","last_became_peered":"2023-06-09T04:35:17.441745+0000","last_unstale":"2023-06-09T04:35:20.243586+0000","last_undegraded":"2023-06-09T04:35:20.243586+0000","last_fullsized":"2023-06-09T04:35:20.243586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:38.026198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354357+0000","last_change":"2023-06-09T04:35:17.440445+0000","last_active":"2023-06-09T04:35:20.354357+0000","last_peered":"2023-06-09T04:35:20.354357+0000","last_clean":"2023-06-09T04:35:20.354357+0000","last_became_active":"2023-06-09T04:35:17.440125+0000","last_became_peered":"2023-06-09T04:35:17.440125+0000","last_unstale":"2023-06-09T04:35:20.354357+0000","last_undegraded":"2023-06-09T04:35:20.354357+0000","last_fullsized":"2023-06-09T04:35:20.354357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423869+0000","last_change":"2023-06-09T04:35:17.442303+0000","last_active":"2023-06-09T04:35:18.423869+0000","last_peered":"2023-06-09T04:35:18.423869+0000","last_clean":"2023-06-09T04:35:18.423869+0000","last_became_active":"2023-06-09T04:35:17.442041+0000","last_became_peered":"2023-06-09T04:35:17.442041+0000","last_unstale":"2023-06-09T04:35:18.423869+0000","last_undegraded":"2023-06-09T04:35:18.423869+0000","last_fullsized":"2023-06-09T04:35:18.423869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243381+0000","last_change":"2023-06-09T04:35:17.436865+0000","last_active":"2023-06-09T04:35:20.243381+0000","last_peered":"2023-06-09T04:35:20.243381+0000","last_clean":"2023-06-09T04:35:20.243381+0000","last_became_active":"2023-06-09T04:35:17.436545+0000","last_became_peered":"2023-06-09T04:35:17.436545+0000","last_unstale":"2023-06-09T04:35:20.243381+0000","last_undegraded":"2023-06-09T04:35:20.243381+0000","last_fullsized":"2023-06-09T04:35:20.243381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353900+0000","last_change":"2023-06-09T04:35:17.442515+0000","last_active":"2023-06-09T04:35:20.353900+0000","last_peered":"2023-06-09T04:35:20.353900+0000","last_clean":"2023-06-09T04:35:20.353900+0000","last_became_active":"2023-06-09T04:35:17.442247+0000","last_became_peered":"2023-06-09T04:35:17.442247+0000","last_unstale":"2023-06-09T04:35:20.353900+0000","last_undegraded":"2023-06-09T04:35:20.353900+0000","last_fullsized":"2023-06-09T04:35:20.353900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354958+0000","last_change":"2023-06-09T04:35:17.435870+0000","last_active":"2023-06-09T04:35:20.354958+0000","last_peered":"2023-06-09T04:35:20.354958+0000","last_clean":"2023-06-09T04:35:20.354958+0000","last_became_active":"2023-06-09T04:35:17.435616+0000","last_became_peered":"2023-06-09T04:35:17.435616+0000","last_unstale":"2023-06-09T04:35:20.354958+0000","last_undegraded":"2023-06-09T04:35:20.354958+0000","last_fullsized":"2023-06-09T04:35:20.354958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:25.978819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424017+0000","last_change":"2023-06-09T04:35:17.438485+0000","last_active":"2023-06-09T04:35:18.424017+0000","last_peered":"2023-06-09T04:35:18.424017+0000","last_clean":"2023-06-09T04:35:18.424017+0000","last_became_active":"2023-06-09T04:35:17.438173+0000","last_became_peered":"2023-06-09T04:35:17.438173+0000","last_unstale":"2023-06-09T04:35:18.424017+0000","last_undegraded":"2023-06-09T04:35:18.424017+0000","last_fullsized":"2023-06-09T04:35:18.424017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:03.444952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071067+0000","last_change":"2023-06-09T04:35:17.032888+0000","last_active":"2023-06-09T04:35:19.071067+0000","last_peered":"2023-06-09T04:35:19.071067+0000","last_clean":"2023-06-09T04:35:19.071067+0000","last_became_active":"2023-06-09T04:35:17.032675+0000","last_became_peered":"2023-06-09T04:35:17.032675+0000","last_unstale":"2023-06-09T04:35:19.071067+0000","last_undegraded":"2023-06-09T04:35:19.071067+0000","last_fullsized":"2023-06-09T04:35:19.071067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:02.400588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424118+0000","last_change":"2023-06-09T04:35:17.439014+0000","last_active":"2023-06-09T04:35:18.424118+0000","last_peered":"2023-06-09T04:35:18.424118+0000","last_clean":"2023-06-09T04:35:18.424118+0000","last_became_active":"2023-06-09T04:35:17.438737+0000","last_became_peered":"2023-06-09T04:35:17.438737+0000","last_unstale":"2023-06-09T04:35:18.424118+0000","last_undegraded":"2023-06-09T04:35:18.424118+0000","last_fullsized":"2023-06-09T04:35:18.424118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:03.406729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353995+0000","last_change":"2023-06-09T04:35:17.029890+0000","last_active":"2023-06-09T04:35:20.353995+0000","last_peered":"2023-06-09T04:35:20.353995+0000","last_clean":"2023-06-09T04:35:20.353995+0000","last_became_active":"2023-06-09T04:35:17.029430+0000","last_became_peered":"2023-06-09T04:35:17.029430+0000","last_unstale":"2023-06-09T04:35:20.353995+0000","last_undegraded":"2023-06-09T04:35:20.353995+0000","last_fullsized":"2023-06-09T04:35:20.353995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:54.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354046+0000","last_change":"2023-06-09T04:35:17.430052+0000","last_active":"2023-06-09T04:35:20.354046+0000","last_peered":"2023-06-09T04:35:20.354046+0000","last_clean":"2023-06-09T04:35:20.354046+0000","last_became_active":"2023-06-09T04:35:17.429777+0000","last_became_peered":"2023-06-09T04:35:17.429777+0000","last_unstale":"2023-06-09T04:35:20.354046+0000","last_undegraded":"2023-06-09T04:35:20.354046+0000","last_fullsized":"2023-06-09T04:35:20.354046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423796+0000","last_change":"2023-06-09T04:35:17.030206+0000","last_active":"2023-06-09T04:35:18.423796+0000","last_peered":"2023-06-09T04:35:18.423796+0000","last_clean":"2023-06-09T04:35:18.423796+0000","last_became_active":"2023-06-09T04:35:17.029820+0000","last_became_peered":"2023-06-09T04:35:17.029820+0000","last_unstale":"2023-06-09T04:35:18.423796+0000","last_undegraded":"2023-06-09T04:35:18.423796+0000","last_fullsized":"2023-06-09T04:35:18.423796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423430+0000","last_change":"2023-06-09T04:35:17.440301+0000","last_active":"2023-06-09T04:35:18.423430+0000","last_peered":"2023-06-09T04:35:18.423430+0000","last_clean":"2023-06-09T04:35:18.423430+0000","last_became_active":"2023-06-09T04:35:17.440026+0000","last_became_peered":"2023-06-09T04:35:17.440026+0000","last_unstale":"2023-06-09T04:35:18.423430+0000","last_undegraded":"2023-06-09T04:35:18.423430+0000","last_fullsized":"2023-06-09T04:35:18.423430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:32:18.360318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354131+0000","last_change":"2023-06-09T04:35:17.029374+0000","last_active":"2023-06-09T04:35:20.354131+0000","last_peered":"2023-06-09T04:35:20.354131+0000","last_clean":"2023-06-09T04:35:20.354131+0000","last_became_active":"2023-06-09T04:35:17.029069+0000","last_became_peered":"2023-06-09T04:35:17.029069+0000","last_unstale":"2023-06-09T04:35:20.354131+0000","last_undegraded":"2023-06-09T04:35:20.354131+0000","last_fullsized":"2023-06-09T04:35:20.354131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423667+0000","last_change":"2023-06-09T04:35:17.436424+0000","last_active":"2023-06-09T04:35:18.423667+0000","last_peered":"2023-06-09T04:35:18.423667+0000","last_clean":"2023-06-09T04:35:18.423667+0000","last_became_active":"2023-06-09T04:35:17.436173+0000","last_became_peered":"2023-06-09T04:35:17.436173+0000","last_unstale":"2023-06-09T04:35:18.423667+0000","last_undegraded":"2023-06-09T04:35:18.423667+0000","last_fullsized":"2023-06-09T04:35:18.423667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:35:03.891489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423864+0000","last_change":"2023-06-09T04:35:17.024674+0000","last_active":"2023-06-09T04:35:18.423864+0000","last_peered":"2023-06-09T04:35:18.423864+0000","last_clean":"2023-06-09T04:35:18.423864+0000","last_became_active":"2023-06-09T04:35:17.024402+0000","last_became_peered":"2023-06-09T04:35:17.024402+0000","last_unstale":"2023-06-09T04:35:18.423864+0000","last_undegraded":"2023-06-09T04:35:18.423864+0000","last_fullsized":"2023-06-09T04:35:18.423864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423738+0000","last_change":"2023-06-09T04:35:17.431496+0000","last_active":"2023-06-09T04:35:18.423738+0000","last_peered":"2023-06-09T04:35:18.423738+0000","last_clean":"2023-06-09T04:35:18.423738+0000","last_became_active":"2023-06-09T04:35:17.431197+0000","last_became_peered":"2023-06-09T04:35:17.431197+0000","last_unstale":"2023-06-09T04:35:18.423738+0000","last_undegraded":"2023-06-09T04:35:18.423738+0000","last_fullsized":"2023-06-09T04:35:18.423738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243596+0000","last_change":"2023-06-09T04:35:17.029451+0000","last_active":"2023-06-09T04:35:20.243596+0000","last_peered":"2023-06-09T04:35:20.243596+0000","last_clean":"2023-06-09T04:35:20.243596+0000","last_became_active":"2023-06-09T04:35:17.029169+0000","last_became_peered":"2023-06-09T04:35:17.029169+0000","last_unstale":"2023-06-09T04:35:20.243596+0000","last_undegraded":"2023-06-09T04:35:20.243596+0000","last_fullsized":"2023-06-09T04:35:20.243596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:14.749659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354582+0000","last_change":"2023-06-09T04:35:17.438193+0000","last_active":"2023-06-09T04:35:20.354582+0000","last_peered":"2023-06-09T04:35:20.354582+0000","last_clean":"2023-06-09T04:35:20.354582+0000","last_became_active":"2023-06-09T04:35:17.437765+0000","last_became_peered":"2023-06-09T04:35:17.437765+0000","last_unstale":"2023-06-09T04:35:20.354582+0000","last_undegraded":"2023-06-09T04:35:20.354582+0000","last_fullsized":"2023-06-09T04:35:20.354582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:03.444952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071305+0000","last_change":"2023-06-09T04:35:17.028442+0000","last_active":"2023-06-09T04:35:19.071305+0000","last_peered":"2023-06-09T04:35:19.071305+0000","last_clean":"2023-06-09T04:35:19.071305+0000","last_became_active":"2023-06-09T04:35:17.028162+0000","last_became_peered":"2023-06-09T04:35:17.028162+0000","last_unstale":"2023-06-09T04:35:19.071305+0000","last_undegraded":"2023-06-09T04:35:19.071305+0000","last_fullsized":"2023-06-09T04:35:19.071305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:37.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071228+0000","last_change":"2023-06-09T04:35:17.433199+0000","last_active":"2023-06-09T04:35:19.071228+0000","last_peered":"2023-06-09T04:35:19.071228+0000","last_clean":"2023-06-09T04:35:19.071228+0000","last_became_active":"2023-06-09T04:35:17.432807+0000","last_became_peered":"2023-06-09T04:35:17.432807+0000","last_unstale":"2023-06-09T04:35:19.071228+0000","last_undegraded":"2023-06-09T04:35:19.071228+0000","last_fullsized":"2023-06-09T04:35:19.071228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:56:00.781717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243460+0000","last_change":"2023-06-09T04:35:17.029495+0000","last_active":"2023-06-09T04:35:20.243460+0000","last_peered":"2023-06-09T04:35:20.243460+0000","last_clean":"2023-06-09T04:35:20.243460+0000","last_became_active":"2023-06-09T04:35:17.028510+0000","last_became_peered":"2023-06-09T04:35:17.028510+0000","last_unstale":"2023-06-09T04:35:20.243460+0000","last_undegraded":"2023-06-09T04:35:20.243460+0000","last_fullsized":"2023-06-09T04:35:20.243460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:24.972678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423724+0000","last_change":"2023-06-09T04:35:17.436310+0000","last_active":"2023-06-09T04:35:18.423724+0000","last_peered":"2023-06-09T04:35:18.423724+0000","last_clean":"2023-06-09T04:35:18.423724+0000","last_became_active":"2023-06-09T04:35:17.436034+0000","last_became_peered":"2023-06-09T04:35:17.436034+0000","last_unstale":"2023-06-09T04:35:18.423724+0000","last_undegraded":"2023-06-09T04:35:18.423724+0000","last_fullsized":"2023-06-09T04:35:18.423724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353670+0000","last_change":"2023-06-09T04:35:17.028971+0000","last_active":"2023-06-09T04:35:20.353670+0000","last_peered":"2023-06-09T04:35:20.353670+0000","last_clean":"2023-06-09T04:35:20.353670+0000","last_became_active":"2023-06-09T04:35:17.028680+0000","last_became_peered":"2023-06-09T04:35:17.028680+0000","last_unstale":"2023-06-09T04:35:20.353670+0000","last_undegraded":"2023-06-09T04:35:20.353670+0000","last_fullsized":"2023-06-09T04:35:20.353670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:29.160371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424134+0000","last_change":"2023-06-09T04:35:17.438269+0000","last_active":"2023-06-09T04:35:18.424134+0000","last_peered":"2023-06-09T04:35:18.424134+0000","last_clean":"2023-06-09T04:35:18.424134+0000","last_became_active":"2023-06-09T04:35:17.437854+0000","last_became_peered":"2023-06-09T04:35:17.437854+0000","last_unstale":"2023-06-09T04:35:18.424134+0000","last_undegraded":"2023-06-09T04:35:18.424134+0000","last_fullsized":"2023-06-09T04:35:18.424134+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243547+0000","last_change":"2023-06-09T04:35:17.028793+0000","last_active":"2023-06-09T04:35:20.243547+0000","last_peered":"2023-06-09T04:35:20.243547+0000","last_clean":"2023-06-09T04:35:20.243547+0000","last_became_active":"2023-06-09T04:35:17.028480+0000","last_became_peered":"2023-06-09T04:35:17.028480+0000","last_unstale":"2023-06-09T04:35:20.243547+0000","last_undegraded":"2023-06-09T04:35:20.243547+0000","last_fullsized":"2023-06-09T04:35:20.243547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:59.775576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354124+0000","last_change":"2023-06-09T04:35:17.427519+0000","last_active":"2023-06-09T04:35:20.354124+0000","last_peered":"2023-06-09T04:35:20.354124+0000","last_clean":"2023-06-09T04:35:20.354124+0000","last_became_active":"2023-06-09T04:35:17.427246+0000","last_became_peered":"2023-06-09T04:35:17.427246+0000","last_unstale":"2023-06-09T04:35:20.354124+0000","last_undegraded":"2023-06-09T04:35:20.354124+0000","last_fullsized":"2023-06-09T04:35:20.354124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423219+0000","last_change":"2023-06-09T04:35:17.030556+0000","last_active":"2023-06-09T04:35:18.423219+0000","last_peered":"2023-06-09T04:35:18.423219+0000","last_clean":"2023-06-09T04:35:18.423219+0000","last_became_active":"2023-06-09T04:35:17.030279+0000","last_became_peered":"2023-06-09T04:35:17.030279+0000","last_unstale":"2023-06-09T04:35:18.423219+0000","last_undegraded":"2023-06-09T04:35:18.423219+0000","last_fullsized":"2023-06-09T04:35:18.423219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:01:59.581621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424220+0000","last_change":"2023-06-09T04:35:17.431297+0000","last_active":"2023-06-09T04:35:18.424220+0000","last_peered":"2023-06-09T04:35:18.424220+0000","last_clean":"2023-06-09T04:35:18.424220+0000","last_became_active":"2023-06-09T04:35:17.431025+0000","last_became_peered":"2023-06-09T04:35:17.431025+0000","last_unstale":"2023-06-09T04:35:18.424220+0000","last_undegraded":"2023-06-09T04:35:18.424220+0000","last_fullsized":"2023-06-09T04:35:18.424220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243918+0000","last_change":"2023-06-09T04:35:17.028833+0000","last_active":"2023-06-09T04:35:20.243918+0000","last_peered":"2023-06-09T04:35:20.243918+0000","last_clean":"2023-06-09T04:35:20.243918+0000","last_became_active":"2023-06-09T04:35:17.028375+0000","last_became_peered":"2023-06-09T04:35:17.028375+0000","last_unstale":"2023-06-09T04:35:20.243918+0000","last_undegraded":"2023-06-09T04:35:20.243918+0000","last_fullsized":"2023-06-09T04:35:20.243918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:44:50.086544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423784+0000","last_change":"2023-06-09T04:35:17.442725+0000","last_active":"2023-06-09T04:35:18.423784+0000","last_peered":"2023-06-09T04:35:18.423784+0000","last_clean":"2023-06-09T04:35:18.423784+0000","last_became_active":"2023-06-09T04:35:17.442455+0000","last_became_peered":"2023-06-09T04:35:17.442455+0000","last_unstale":"2023-06-09T04:35:18.423784+0000","last_undegraded":"2023-06-09T04:35:18.423784+0000","last_fullsized":"2023-06-09T04:35:18.423784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243859+0000","last_change":"2023-06-09T04:35:17.029278+0000","last_active":"2023-06-09T04:35:20.243859+0000","last_peered":"2023-06-09T04:35:20.243859+0000","last_clean":"2023-06-09T04:35:20.243859+0000","last_became_active":"2023-06-09T04:35:17.028978+0000","last_became_peered":"2023-06-09T04:35:17.028978+0000","last_unstale":"2023-06-09T04:35:20.243859+0000","last_undegraded":"2023-06-09T04:35:20.243859+0000","last_fullsized":"2023-06-09T04:35:20.243859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:17:16.655161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070861+0000","last_change":"2023-06-09T04:35:17.438515+0000","last_active":"2023-06-09T04:35:19.070861+0000","last_peered":"2023-06-09T04:35:19.070861+0000","last_clean":"2023-06-09T04:35:19.070861+0000","last_became_active":"2023-06-09T04:35:17.437901+0000","last_became_peered":"2023-06-09T04:35:17.437901+0000","last_unstale":"2023-06-09T04:35:19.070861+0000","last_undegraded":"2023-06-09T04:35:19.070861+0000","last_fullsized":"2023-06-09T04:35:19.070861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243775+0000","last_change":"2023-06-09T04:35:17.028112+0000","last_active":"2023-06-09T04:35:20.243775+0000","last_peered":"2023-06-09T04:35:20.243775+0000","last_clean":"2023-06-09T04:35:20.243775+0000","last_became_active":"2023-06-09T04:35:17.027836+0000","last_became_peered":"2023-06-09T04:35:17.027836+0000","last_unstale":"2023-06-09T04:35:20.243775+0000","last_undegraded":"2023-06-09T04:35:20.243775+0000","last_fullsized":"2023-06-09T04:35:20.243775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:25:10.314437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423291+0000","last_change":"2023-06-09T04:35:17.436148+0000","last_active":"2023-06-09T04:35:18.423291+0000","last_peered":"2023-06-09T04:35:18.423291+0000","last_clean":"2023-06-09T04:35:18.423291+0000","last_became_active":"2023-06-09T04:35:17.435824+0000","last_became_peered":"2023-06-09T04:35:17.435824+0000","last_unstale":"2023-06-09T04:35:18.423291+0000","last_undegraded":"2023-06-09T04:35:18.423291+0000","last_fullsized":"2023-06-09T04:35:18.423291+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:10.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.425087+0000","last_change":"2023-06-09T04:35:17.026454+0000","last_active":"2023-06-09T04:35:18.425087+0000","last_peered":"2023-06-09T04:35:18.425087+0000","last_clean":"2023-06-09T04:35:18.425087+0000","last_became_active":"2023-06-09T04:35:17.025128+0000","last_became_peered":"2023-06-09T04:35:17.025128+0000","last_unstale":"2023-06-09T04:35:18.425087+0000","last_undegraded":"2023-06-09T04:35:18.425087+0000","last_fullsized":"2023-06-09T04:35:18.425087+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:37.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354445+0000","last_change":"2023-06-09T04:35:17.436924+0000","last_active":"2023-06-09T04:35:20.354445+0000","last_peered":"2023-06-09T04:35:20.354445+0000","last_clean":"2023-06-09T04:35:20.354445+0000","last_became_active":"2023-06-09T04:35:17.436657+0000","last_became_peered":"2023-06-09T04:35:17.436657+0000","last_unstale":"2023-06-09T04:35:20.354445+0000","last_undegraded":"2023-06-09T04:35:20.354445+0000","last_fullsized":"2023-06-09T04:35:20.354445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:03.406729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423986+0000","last_change":"2023-06-09T04:35:17.031120+0000","last_active":"2023-06-09T04:35:18.423986+0000","last_peered":"2023-06-09T04:35:18.423986+0000","last_clean":"2023-06-09T04:35:18.423986+0000","last_became_active":"2023-06-09T04:35:17.030888+0000","last_became_peered":"2023-06-09T04:35:17.030888+0000","last_unstale":"2023-06-09T04:35:18.423986+0000","last_undegraded":"2023-06-09T04:35:18.423986+0000","last_fullsized":"2023-06-09T04:35:18.423986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:05.224171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071228+0000","last_change":"2023-06-09T04:35:17.432702+0000","last_active":"2023-06-09T04:35:19.071228+0000","last_peered":"2023-06-09T04:35:19.071228+0000","last_clean":"2023-06-09T04:35:19.071228+0000","last_became_active":"2023-06-09T04:35:17.432343+0000","last_became_peered":"2023-06-09T04:35:17.432343+0000","last_unstale":"2023-06-09T04:35:19.071228+0000","last_undegraded":"2023-06-09T04:35:19.071228+0000","last_fullsized":"2023-06-09T04:35:19.071228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:25.978819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423112+0000","last_change":"2023-06-09T04:35:17.028382+0000","last_active":"2023-06-09T04:35:18.423112+0000","last_peered":"2023-06-09T04:35:18.423112+0000","last_clean":"2023-06-09T04:35:18.423112+0000","last_became_active":"2023-06-09T04:35:17.028097+0000","last_became_peered":"2023-06-09T04:35:17.028097+0000","last_unstale":"2023-06-09T04:35:18.423112+0000","last_undegraded":"2023-06-09T04:35:18.423112+0000","last_fullsized":"2023-06-09T04:35:18.423112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:57.406268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071469+0000","last_change":"2023-06-09T04:35:17.434946+0000","last_active":"2023-06-09T04:35:19.071469+0000","last_peered":"2023-06-09T04:35:19.071469+0000","last_clean":"2023-06-09T04:35:19.071469+0000","last_became_active":"2023-06-09T04:35:17.434618+0000","last_became_peered":"2023-06-09T04:35:17.434618+0000","last_unstale":"2023-06-09T04:35:19.071469+0000","last_undegraded":"2023-06-09T04:35:19.071469+0000","last_fullsized":"2023-06-09T04:35:19.071469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:07:19.431879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071066+0000","last_change":"2023-06-09T04:35:17.023550+0000","last_active":"2023-06-09T04:35:19.071066+0000","last_peered":"2023-06-09T04:35:19.071066+0000","last_clean":"2023-06-09T04:35:19.071066+0000","last_became_active":"2023-06-09T04:35:17.022566+0000","last_became_peered":"2023-06-09T04:35:17.022566+0000","last_unstale":"2023-06-09T04:35:19.071066+0000","last_undegraded":"2023-06-09T04:35:19.071066+0000","last_fullsized":"2023-06-09T04:35:19.071066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071502+0000","last_change":"2023-06-09T04:35:17.431463+0000","last_active":"2023-06-09T04:35:19.071502+0000","last_peered":"2023-06-09T04:35:19.071502+0000","last_clean":"2023-06-09T04:35:19.071502+0000","last_became_active":"2023-06-09T04:35:17.431065+0000","last_became_peered":"2023-06-09T04:35:17.431065+0000","last_unstale":"2023-06-09T04:35:19.071502+0000","last_undegraded":"2023-06-09T04:35:19.071502+0000","last_fullsized":"2023-06-09T04:35:19.071502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:53:57.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423829+0000","last_change":"2023-06-09T04:35:17.032982+0000","last_active":"2023-06-09T04:35:18.423829+0000","last_peered":"2023-06-09T04:35:18.423829+0000","last_clean":"2023-06-09T04:35:18.423829+0000","last_became_active":"2023-06-09T04:35:17.032679+0000","last_became_peered":"2023-06-09T04:35:17.032679+0000","last_unstale":"2023-06-09T04:35:18.423829+0000","last_undegraded":"2023-06-09T04:35:18.423829+0000","last_fullsized":"2023-06-09T04:35:18.423829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:54.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354242+0000","last_change":"2023-06-09T04:35:17.442076+0000","last_active":"2023-06-09T04:35:20.354242+0000","last_peered":"2023-06-09T04:35:20.354242+0000","last_clean":"2023-06-09T04:35:20.354242+0000","last_became_active":"2023-06-09T04:35:17.441806+0000","last_became_peered":"2023-06-09T04:35:17.441806+0000","last_unstale":"2023-06-09T04:35:20.354242+0000","last_undegraded":"2023-06-09T04:35:20.354242+0000","last_fullsized":"2023-06-09T04:35:20.354242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354557+0000","last_change":"2023-06-09T04:35:12.072943+0000","last_active":"2023-06-09T04:35:20.354557+0000","last_peered":"2023-06-09T04:35:20.354557+0000","last_clean":"2023-06-09T04:35:20.354557+0000","last_became_active":"2023-06-09T04:35:12.072623+0000","last_became_peered":"2023-06-09T04:35:12.072623+0000","last_unstale":"2023-06-09T04:35:20.354557+0000","last_undegraded":"2023-06-09T04:35:20.354557+0000","last_fullsized":"2023-06-09T04:35:20.354557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:00.202928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424301+0000","last_change":"2023-06-09T04:35:17.026144+0000","last_active":"2023-06-09T04:35:18.424301+0000","last_peered":"2023-06-09T04:35:18.424301+0000","last_clean":"2023-06-09T04:35:18.424301+0000","last_became_active":"2023-06-09T04:35:17.025699+0000","last_became_peered":"2023-06-09T04:35:17.025699+0000","last_unstale":"2023-06-09T04:35:18.424301+0000","last_undegraded":"2023-06-09T04:35:18.424301+0000","last_fullsized":"2023-06-09T04:35:18.424301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354751+0000","last_change":"2023-06-09T04:35:17.440088+0000","last_active":"2023-06-09T04:35:20.354751+0000","last_peered":"2023-06-09T04:35:20.354751+0000","last_clean":"2023-06-09T04:35:20.354751+0000","last_became_active":"2023-06-09T04:35:17.439771+0000","last_became_peered":"2023-06-09T04:35:17.439771+0000","last_unstale":"2023-06-09T04:35:20.354751+0000","last_undegraded":"2023-06-09T04:35:20.354751+0000","last_fullsized":"2023-06-09T04:35:20.354751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424339+0000","last_change":"2023-06-09T04:35:12.002754+0000","last_active":"2023-06-09T04:35:18.424339+0000","last_peered":"2023-06-09T04:35:18.424339+0000","last_clean":"2023-06-09T04:35:18.424339+0000","last_became_active":"2023-06-09T04:35:12.002471+0000","last_became_peered":"2023-06-09T04:35:12.002471+0000","last_unstale":"2023-06-09T04:35:18.424339+0000","last_undegraded":"2023-06-09T04:35:18.424339+0000","last_fullsized":"2023-06-09T04:35:18.424339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:54.754333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071063+0000","last_change":"2023-06-09T04:35:17.032034+0000","last_active":"2023-06-09T04:35:19.071063+0000","last_peered":"2023-06-09T04:35:19.071063+0000","last_clean":"2023-06-09T04:35:19.071063+0000","last_became_active":"2023-06-09T04:35:17.031771+0000","last_became_peered":"2023-06-09T04:35:17.031771+0000","last_unstale":"2023-06-09T04:35:19.071063+0000","last_undegraded":"2023-06-09T04:35:19.071063+0000","last_fullsized":"2023-06-09T04:35:19.071063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:53.206518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354666+0000","last_change":"2023-06-09T04:35:17.439779+0000","last_active":"2023-06-09T04:35:20.354666+0000","last_peered":"2023-06-09T04:35:20.354666+0000","last_clean":"2023-06-09T04:35:20.354666+0000","last_became_active":"2023-06-09T04:35:17.439539+0000","last_became_peered":"2023-06-09T04:35:17.439539+0000","last_unstale":"2023-06-09T04:35:20.354666+0000","last_undegraded":"2023-06-09T04:35:20.354666+0000","last_fullsized":"2023-06-09T04:35:20.354666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:03.406729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955491+0000","last_change":"2023-06-09T04:35:12.071356+0000","last_active":"2023-06-09T04:35:18.955491+0000","last_peered":"2023-06-09T04:35:18.955491+0000","last_clean":"2023-06-09T04:35:18.955491+0000","last_became_active":"2023-06-09T04:35:12.070997+0000","last_became_peered":"2023-06-09T04:35:12.070997+0000","last_unstale":"2023-06-09T04:35:18.955491+0000","last_undegraded":"2023-06-09T04:35:18.955491+0000","last_fullsized":"2023-06-09T04:35:18.955491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:52.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354331+0000","last_change":"2023-06-09T04:35:17.027888+0000","last_active":"2023-06-09T04:35:20.354331+0000","last_peered":"2023-06-09T04:35:20.354331+0000","last_clean":"2023-06-09T04:35:20.354331+0000","last_became_active":"2023-06-09T04:35:17.027381+0000","last_became_peered":"2023-06-09T04:35:17.027381+0000","last_unstale":"2023-06-09T04:35:20.354331+0000","last_undegraded":"2023-06-09T04:35:20.354331+0000","last_fullsized":"2023-06-09T04:35:20.354331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354038+0000","last_change":"2023-06-09T04:35:17.444159+0000","last_active":"2023-06-09T04:35:20.354038+0000","last_peered":"2023-06-09T04:35:20.354038+0000","last_clean":"2023-06-09T04:35:20.354038+0000","last_became_active":"2023-06-09T04:35:17.443862+0000","last_became_peered":"2023-06-09T04:35:17.443862+0000","last_unstale":"2023-06-09T04:35:20.354038+0000","last_undegraded":"2023-06-09T04:35:20.354038+0000","last_fullsized":"2023-06-09T04:35:20.354038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424131+0000","last_change":"2023-06-09T04:35:11.999027+0000","last_active":"2023-06-09T04:35:18.424131+0000","last_peered":"2023-06-09T04:35:18.424131+0000","last_clean":"2023-06-09T04:35:18.424131+0000","last_became_active":"2023-06-09T04:35:11.998723+0000","last_became_peered":"2023-06-09T04:35:11.998723+0000","last_unstale":"2023-06-09T04:35:18.424131+0000","last_undegraded":"2023-06-09T04:35:18.424131+0000","last_fullsized":"2023-06-09T04:35:18.424131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:40:57.379345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243437+0000","last_change":"2023-06-09T04:35:06.996716+0000","last_active":"2023-06-09T04:35:20.243437+0000","last_peered":"2023-06-09T04:35:20.243437+0000","last_clean":"2023-06-09T04:35:20.243437+0000","last_became_active":"2023-06-09T04:35:06.995565+0000","last_became_peered":"2023-06-09T04:35:06.995565+0000","last_unstale":"2023-06-09T04:35:20.243437+0000","last_undegraded":"2023-06-09T04:35:20.243437+0000","last_fullsized":"2023-06-09T04:35:20.243437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_clean_scrub_stamp":"2023-06-09T04:35:05.906836+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:23:53.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423735+0000","last_change":"2023-06-09T04:35:17.019063+0000","last_active":"2023-06-09T04:35:18.423735+0000","last_peered":"2023-06-09T04:35:18.423735+0000","last_clean":"2023-06-09T04:35:18.423735+0000","last_became_active":"2023-06-09T04:35:17.018762+0000","last_became_peered":"2023-06-09T04:35:17.018762+0000","last_unstale":"2023-06-09T04:35:18.423735+0000","last_undegraded":"2023-06-09T04:35:18.423735+0000","last_fullsized":"2023-06-09T04:35:18.423735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:29.160371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423249+0000","last_change":"2023-06-09T04:35:17.445339+0000","last_active":"2023-06-09T04:35:18.423249+0000","last_peered":"2023-06-09T04:35:18.423249+0000","last_clean":"2023-06-09T04:35:18.423249+0000","last_became_active":"2023-06-09T04:35:17.445113+0000","last_became_peered":"2023-06-09T04:35:17.445113+0000","last_unstale":"2023-06-09T04:35:18.423249+0000","last_undegraded":"2023-06-09T04:35:18.423249+0000","last_fullsized":"2023-06-09T04:35:18.423249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243611+0000","last_change":"2023-06-09T04:35:12.002654+0000","last_active":"2023-06-09T04:35:20.243611+0000","last_peered":"2023-06-09T04:35:20.243611+0000","last_clean":"2023-06-09T04:35:20.243611+0000","last_became_active":"2023-06-09T04:35:12.002320+0000","last_became_peered":"2023-06-09T04:35:12.002320+0000","last_unstale":"2023-06-09T04:35:20.243611+0000","last_undegraded":"2023-06-09T04:35:20.243611+0000","last_fullsized":"2023-06-09T04:35:20.243611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:58:21.712140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353644+0000","last_change":"2023-06-09T04:35:17.029537+0000","last_active":"2023-06-09T04:35:20.353644+0000","last_peered":"2023-06-09T04:35:20.353644+0000","last_clean":"2023-06-09T04:35:20.353644+0000","last_became_active":"2023-06-09T04:35:17.029185+0000","last_became_peered":"2023-06-09T04:35:17.029185+0000","last_unstale":"2023-06-09T04:35:20.353644+0000","last_undegraded":"2023-06-09T04:35:20.353644+0000","last_fullsized":"2023-06-09T04:35:20.353644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:38:43.929108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353900+0000","last_change":"2023-06-09T04:35:17.435253+0000","last_active":"2023-06-09T04:35:20.353900+0000","last_peered":"2023-06-09T04:35:20.353900+0000","last_clean":"2023-06-09T04:35:20.353900+0000","last_became_active":"2023-06-09T04:35:17.434964+0000","last_became_peered":"2023-06-09T04:35:17.434964+0000","last_unstale":"2023-06-09T04:35:20.353900+0000","last_undegraded":"2023-06-09T04:35:20.353900+0000","last_fullsized":"2023-06-09T04:35:20.353900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:55.606991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243708+0000","last_change":"2023-06-09T04:35:12.071967+0000","last_active":"2023-06-09T04:35:20.243708+0000","last_peered":"2023-06-09T04:35:20.243708+0000","last_clean":"2023-06-09T04:35:20.243708+0000","last_became_active":"2023-06-09T04:35:12.071565+0000","last_became_peered":"2023-06-09T04:35:12.071565+0000","last_unstale":"2023-06-09T04:35:20.243708+0000","last_undegraded":"2023-06-09T04:35:20.243708+0000","last_fullsized":"2023-06-09T04:35:20.243708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:07:13.404494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354701+0000","last_change":"2023-06-09T04:35:17.027679+0000","last_active":"2023-06-09T04:35:20.354701+0000","last_peered":"2023-06-09T04:35:20.354701+0000","last_clean":"2023-06-09T04:35:20.354701+0000","last_became_active":"2023-06-09T04:35:17.027357+0000","last_became_peered":"2023-06-09T04:35:17.027357+0000","last_unstale":"2023-06-09T04:35:20.354701+0000","last_undegraded":"2023-06-09T04:35:20.354701+0000","last_fullsized":"2023-06-09T04:35:20.354701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354195+0000","last_change":"2023-06-09T04:35:17.440902+0000","last_active":"2023-06-09T04:35:20.354195+0000","last_peered":"2023-06-09T04:35:20.354195+0000","last_clean":"2023-06-09T04:35:20.354195+0000","last_became_active":"2023-06-09T04:35:17.440622+0000","last_became_peered":"2023-06-09T04:35:17.440622+0000","last_unstale":"2023-06-09T04:35:20.354195+0000","last_undegraded":"2023-06-09T04:35:20.354195+0000","last_fullsized":"2023-06-09T04:35:20.354195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424216+0000","last_change":"2023-06-09T04:35:17.026812+0000","last_active":"2023-06-09T04:35:18.424216+0000","last_peered":"2023-06-09T04:35:18.424216+0000","last_clean":"2023-06-09T04:35:18.424216+0000","last_became_active":"2023-06-09T04:35:17.026394+0000","last_became_peered":"2023-06-09T04:35:17.026394+0000","last_unstale":"2023-06-09T04:35:18.424216+0000","last_undegraded":"2023-06-09T04:35:18.424216+0000","last_fullsized":"2023-06-09T04:35:18.424216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354610+0000","last_change":"2023-06-09T04:35:12.003438+0000","last_active":"2023-06-09T04:35:20.354610+0000","last_peered":"2023-06-09T04:35:20.354610+0000","last_clean":"2023-06-09T04:35:20.354610+0000","last_became_active":"2023-06-09T04:35:12.003155+0000","last_became_peered":"2023-06-09T04:35:12.003155+0000","last_unstale":"2023-06-09T04:35:20.354610+0000","last_undegraded":"2023-06-09T04:35:20.354610+0000","last_fullsized":"2023-06-09T04:35:20.354610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:48.185275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423524+0000","last_change":"2023-06-09T04:35:17.443931+0000","last_active":"2023-06-09T04:35:18.423524+0000","last_peered":"2023-06-09T04:35:18.423524+0000","last_clean":"2023-06-09T04:35:18.423524+0000","last_became_active":"2023-06-09T04:35:17.443663+0000","last_became_peered":"2023-06-09T04:35:17.443663+0000","last_unstale":"2023-06-09T04:35:18.423524+0000","last_undegraded":"2023-06-09T04:35:18.423524+0000","last_fullsized":"2023-06-09T04:35:18.423524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:43:45.048336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423304+0000","last_change":"2023-06-09T04:35:12.003198+0000","last_active":"2023-06-09T04:35:18.423304+0000","last_peered":"2023-06-09T04:35:18.423304+0000","last_clean":"2023-06-09T04:35:18.423304+0000","last_became_active":"2023-06-09T04:35:12.002905+0000","last_became_peered":"2023-06-09T04:35:12.002905+0000","last_unstale":"2023-06-09T04:35:18.423304+0000","last_undegraded":"2023-06-09T04:35:18.423304+0000","last_fullsized":"2023-06-09T04:35:18.423304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:27:49.237891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354416+0000","last_change":"2023-06-09T04:35:17.033086+0000","last_active":"2023-06-09T04:35:20.354416+0000","last_peered":"2023-06-09T04:35:20.354416+0000","last_clean":"2023-06-09T04:35:20.354416+0000","last_became_active":"2023-06-09T04:35:17.032814+0000","last_became_peered":"2023-06-09T04:35:17.032814+0000","last_unstale":"2023-06-09T04:35:20.354416+0000","last_undegraded":"2023-06-09T04:35:20.354416+0000","last_fullsized":"2023-06-09T04:35:20.354416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070930+0000","last_change":"2023-06-09T04:35:17.438683+0000","last_active":"2023-06-09T04:35:19.070930+0000","last_peered":"2023-06-09T04:35:19.070930+0000","last_clean":"2023-06-09T04:35:19.070930+0000","last_became_active":"2023-06-09T04:35:17.438261+0000","last_became_peered":"2023-06-09T04:35:17.438261+0000","last_unstale":"2023-06-09T04:35:19.070930+0000","last_undegraded":"2023-06-09T04:35:19.070930+0000","last_fullsized":"2023-06-09T04:35:19.070930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:58.758092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354316+0000","last_change":"2023-06-09T04:35:17.031891+0000","last_active":"2023-06-09T04:35:20.354316+0000","last_peered":"2023-06-09T04:35:20.354316+0000","last_clean":"2023-06-09T04:35:20.354316+0000","last_became_active":"2023-06-09T04:35:17.031613+0000","last_became_peered":"2023-06-09T04:35:17.031613+0000","last_unstale":"2023-06-09T04:35:20.354316+0000","last_undegraded":"2023-06-09T04:35:20.354316+0000","last_fullsized":"2023-06-09T04:35:20.354316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955690+0000","last_change":"2023-06-09T04:35:17.437403+0000","last_active":"2023-06-09T04:35:18.955690+0000","last_peered":"2023-06-09T04:35:18.955690+0000","last_clean":"2023-06-09T04:35:18.955690+0000","last_became_active":"2023-06-09T04:35:17.437106+0000","last_became_peered":"2023-06-09T04:35:17.437106+0000","last_unstale":"2023-06-09T04:35:18.955690+0000","last_undegraded":"2023-06-09T04:35:18.955690+0000","last_fullsized":"2023-06-09T04:35:18.955690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:10.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424990+0000","last_change":"2023-06-09T04:35:17.025288+0000","last_active":"2023-06-09T04:35:18.424990+0000","last_peered":"2023-06-09T04:35:18.424990+0000","last_clean":"2023-06-09T04:35:18.424990+0000","last_became_active":"2023-06-09T04:35:17.024862+0000","last_became_peered":"2023-06-09T04:35:17.024862+0000","last_unstale":"2023-06-09T04:35:18.424990+0000","last_undegraded":"2023-06-09T04:35:18.424990+0000","last_fullsized":"2023-06-09T04:35:18.424990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:00.851254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070993+0000","last_change":"2023-06-09T04:35:17.434826+0000","last_active":"2023-06-09T04:35:19.070993+0000","last_peered":"2023-06-09T04:35:19.070993+0000","last_clean":"2023-06-09T04:35:19.070993+0000","last_became_active":"2023-06-09T04:35:17.433157+0000","last_became_peered":"2023-06-09T04:35:17.433157+0000","last_unstale":"2023-06-09T04:35:19.070993+0000","last_undegraded":"2023-06-09T04:35:19.070993+0000","last_fullsized":"2023-06-09T04:35:19.070993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424370+0000","last_change":"2023-06-09T04:35:17.025975+0000","last_active":"2023-06-09T04:35:18.424370+0000","last_peered":"2023-06-09T04:35:18.424370+0000","last_clean":"2023-06-09T04:35:18.424370+0000","last_became_active":"2023-06-09T04:35:17.025358+0000","last_became_peered":"2023-06-09T04:35:17.025358+0000","last_unstale":"2023-06-09T04:35:18.424370+0000","last_undegraded":"2023-06-09T04:35:18.424370+0000","last_fullsized":"2023-06-09T04:35:18.424370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:24.972678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071194+0000","last_change":"2023-06-09T04:35:17.435124+0000","last_active":"2023-06-09T04:35:19.071194+0000","last_peered":"2023-06-09T04:35:19.071194+0000","last_clean":"2023-06-09T04:35:19.071194+0000","last_became_active":"2023-06-09T04:35:17.434182+0000","last_became_peered":"2023-06-09T04:35:17.434182+0000","last_unstale":"2023-06-09T04:35:19.071194+0000","last_undegraded":"2023-06-09T04:35:19.071194+0000","last_fullsized":"2023-06-09T04:35:19.071194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424031+0000","last_change":"2023-06-09T04:35:17.025538+0000","last_active":"2023-06-09T04:35:18.424031+0000","last_peered":"2023-06-09T04:35:18.424031+0000","last_clean":"2023-06-09T04:35:18.424031+0000","last_became_active":"2023-06-09T04:35:17.025170+0000","last_became_peered":"2023-06-09T04:35:17.025170+0000","last_unstale":"2023-06-09T04:35:18.424031+0000","last_undegraded":"2023-06-09T04:35:18.424031+0000","last_fullsized":"2023-06-09T04:35:18.424031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423757+0000","last_change":"2023-06-09T04:35:17.437701+0000","last_active":"2023-06-09T04:35:18.423757+0000","last_peered":"2023-06-09T04:35:18.423757+0000","last_clean":"2023-06-09T04:35:18.423757+0000","last_became_active":"2023-06-09T04:35:17.437342+0000","last_became_peered":"2023-06-09T04:35:17.437342+0000","last_unstale":"2023-06-09T04:35:18.423757+0000","last_undegraded":"2023-06-09T04:35:18.423757+0000","last_fullsized":"2023-06-09T04:35:18.423757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354858+0000","last_change":"2023-06-09T04:35:17.029009+0000","last_active":"2023-06-09T04:35:20.354858+0000","last_peered":"2023-06-09T04:35:20.354858+0000","last_clean":"2023-06-09T04:35:20.354858+0000","last_became_active":"2023-06-09T04:35:17.028662+0000","last_became_peered":"2023-06-09T04:35:17.028662+0000","last_unstale":"2023-06-09T04:35:20.354858+0000","last_undegraded":"2023-06-09T04:35:20.354858+0000","last_fullsized":"2023-06-09T04:35:20.354858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:34.490265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070711+0000","last_change":"2023-06-09T04:35:17.434890+0000","last_active":"2023-06-09T04:35:19.070711+0000","last_peered":"2023-06-09T04:35:19.070711+0000","last_clean":"2023-06-09T04:35:19.070711+0000","last_became_active":"2023-06-09T04:35:17.434589+0000","last_became_peered":"2023-06-09T04:35:17.434589+0000","last_unstale":"2023-06-09T04:35:19.070711+0000","last_undegraded":"2023-06-09T04:35:19.070711+0000","last_fullsized":"2023-06-09T04:35:19.070711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423112+0000","last_change":"2023-06-09T04:35:17.431610+0000","last_active":"2023-06-09T04:35:18.423112+0000","last_peered":"2023-06-09T04:35:18.423112+0000","last_clean":"2023-06-09T04:35:18.423112+0000","last_became_active":"2023-06-09T04:35:17.431333+0000","last_became_peered":"2023-06-09T04:35:17.431333+0000","last_unstale":"2023-06-09T04:35:18.423112+0000","last_undegraded":"2023-06-09T04:35:18.423112+0000","last_fullsized":"2023-06-09T04:35:18.423112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423602+0000","last_change":"2023-06-09T04:35:17.435324+0000","last_active":"2023-06-09T04:35:18.423602+0000","last_peered":"2023-06-09T04:35:18.423602+0000","last_clean":"2023-06-09T04:35:18.423602+0000","last_became_active":"2023-06-09T04:35:17.435058+0000","last_became_peered":"2023-06-09T04:35:17.435058+0000","last_unstale":"2023-06-09T04:35:18.423602+0000","last_undegraded":"2023-06-09T04:35:18.423602+0000","last_fullsized":"2023-06-09T04:35:18.423602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:38:44.935250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070935+0000","last_change":"2023-06-09T04:35:17.434438+0000","last_active":"2023-06-09T04:35:19.070935+0000","last_peered":"2023-06-09T04:35:19.070935+0000","last_clean":"2023-06-09T04:35:19.070935+0000","last_became_active":"2023-06-09T04:35:17.432923+0000","last_became_peered":"2023-06-09T04:35:17.432923+0000","last_unstale":"2023-06-09T04:35:19.070935+0000","last_undegraded":"2023-06-09T04:35:19.070935+0000","last_fullsized":"2023-06-09T04:35:19.070935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424114+0000","last_change":"2023-06-09T04:35:17.431908+0000","last_active":"2023-06-09T04:35:18.424114+0000","last_peered":"2023-06-09T04:35:18.424114+0000","last_clean":"2023-06-09T04:35:18.424114+0000","last_became_active":"2023-06-09T04:35:17.431682+0000","last_became_peered":"2023-06-09T04:35:17.431682+0000","last_unstale":"2023-06-09T04:35:18.424114+0000","last_undegraded":"2023-06-09T04:35:18.424114+0000","last_fullsized":"2023-06-09T04:35:18.424114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354496+0000","last_change":"2023-06-09T04:35:17.435983+0000","last_active":"2023-06-09T04:35:20.354496+0000","last_peered":"2023-06-09T04:35:20.354496+0000","last_clean":"2023-06-09T04:35:20.354496+0000","last_became_active":"2023-06-09T04:35:17.435723+0000","last_became_peered":"2023-06-09T04:35:17.435723+0000","last_unstale":"2023-06-09T04:35:20.354496+0000","last_undegraded":"2023-06-09T04:35:20.354496+0000","last_fullsized":"2023-06-09T04:35:20.354496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":686418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4449999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":684551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":94360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":95890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":94922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T04:35:26.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T04:35:26.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.276+0000 7fc0a5819700 1 Processor -- start 2023-06-09T04:35:26.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.276+0000 7fc0a5819700 1 -- start start 2023-06-09T04:35:26.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc0a5819700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc0a00b2070 0x7fc0a00ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc0a5819700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc0a00cd1a0 0x7fc0a00cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc0a5819700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a00d17c0 con 0x7fc0a00cfa70 2023-06-09T04:35:26.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc0a5819700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a00cff10 con 0x7fc0a00cd1a0 2023-06-09T04:35:26.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc0a5819700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a00d0720 con 0x7fc0a00b2070 2023-06-09T04:35:26.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc0a00cd1a0 0x7fc0a00cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc0a00cd1a0 0x7fc0a00cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38000/0 (socket says 172.21.15.92:38000) 2023-06-09T04:35:26.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 -- 172.21.15.92:0/3414047261 learned_addr learned my addr 172.21.15.92:0/3414047261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc0a00cfa70 0x7fc0a00d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36334/0 (socket says 172.21.15.92:36334) 2023-06-09T04:35:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09effd700 1 --2- 172.21.15.92:0/3414047261 >> v2:172.21.15.123:3300/0 conn(0x7fc0a00b2070 0x7fc0a00ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 -- 172.21.15.92:0/3414047261 >> v2:172.21.15.123:3300/0 conn(0x7fc0a00b2070 msgr2=0x7fc0a00ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 --2- 172.21.15.92:0/3414047261 >> v2:172.21.15.123:3300/0 conn(0x7fc0a00b2070 0x7fc0a00ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 -- 172.21.15.92:0/3414047261 >> v1:172.21.15.92:6789/0 conn(0x7fc0a00cfa70 legacy=0x7fc0a00d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09dffb700 1 -- 172.21.15.92:0/3414047261 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251403066 0 0) 0x7fc098001490 con 0x7fc0a00cfa70 2023-06-09T04:35:26.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 -- 172.21.15.92:0/3414047261 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a00d0a00 con 0x7fc0a00cd1a0 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09e7fc700 1 --2- 172.21.15.92:0/3414047261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a00cf530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0a006cc40 tx=0x7fc090009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370f2cf5ae659ce3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09dffb700 1 -- 172.21.15.92:0/3414047261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000b870 con 0x7fc0a00cd1a0 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.277+0000 7fc09dffb700 1 -- 172.21.15.92:0/3414047261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09000b9d0 con 0x7fc0a00cd1a0 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc09dffb700 1 -- 172.21.15.92:0/3414047261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000bd60 con 0x7fc0a00cd1a0 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 -- 172.21.15.92:0/3414047261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00cd1a0 msgr2=0x7fc0a00cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3414047261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a00cf530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc0a006cc40 tx=0x7fc090009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 -- 172.21.15.92:0/3414047261 shutdown_connections 2023-06-09T04:35:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3414047261 >> v2:172.21.15.123:3300/0 conn(0x7fc0a00b2070 0x7fc0a00ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3414047261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a00cf530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 -- 172.21.15.92:0/3414047261 >> 172.21.15.92:0/3414047261 conn(0x7fc0a0065a50 msgr2=0x7fc0a0065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 -- 172.21.15.92:0/3414047261 shutdown_connections 2023-06-09T04:35:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.278+0000 7fc0a5819700 1 -- 172.21.15.92:0/3414047261 wait complete. 2023-06-09T04:35:26.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.279+0000 7fc0a5819700 1 Processor -- start 2023-06-09T04:35:26.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.279+0000 7fc0a5819700 1 -- start start 2023-06-09T04:35:26.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc0a5819700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00b2070 0x7fc0a015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc0a5819700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a01e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc0a5819700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a00cfa70 0x7fc0a01e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc0a5819700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01e8250 con 0x7fc0a00b2070 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc0a5819700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a015b070 con 0x7fc0a00cd1a0 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc0a5819700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a01e86e0 con 0x7fc0a00cfa70 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc09effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00b2070 0x7fc0a015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc09effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00b2070 0x7fc0a015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38004/0 (socket says 172.21.15.92:38004) 2023-06-09T04:35:26.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc09effd700 1 -- 172.21.15.92:0/3370930498 learned_addr learned my addr 172.21.15.92:0/3370930498 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc09e7fc700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a01e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc09effd700 1 -- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a00cfa70 msgr2=0x7fc0a01e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.280+0000 7fc09f7fe700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a00cfa70 0x7fc0a01e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc09effd700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a00cfa70 0x7fc0a01e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc09effd700 1 -- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a00cd1a0 msgr2=0x7fc0a01e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc09effd700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a01e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc09effd700 1 -- 172.21.15.92:0/3370930498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc090009550 con 0x7fc0a00b2070 2023-06-09T04:35:26.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc09e7fc700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a01e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc09f7fe700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a00cfa70 0x7fc0a01e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc09effd700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00b2070 0x7fc0a015a3a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc09400e840 tx=0x7fc09400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc087fff700 1 -- 172.21.15.92:0/3370930498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094013610 con 0x7fc0a00b2070 2023-06-09T04:35:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc087fff700 1 -- 172.21.15.92:0/3370930498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094013770 con 0x7fc0a00b2070 2023-06-09T04:35:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a00c7920 con 0x7fc0a00b2070 2023-06-09T04:35:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc087fff700 1 -- 172.21.15.92:0/3370930498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094013b20 con 0x7fc0a00b2070 2023-06-09T04:35:26.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.281+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a01e7920 con 0x7fc0a00b2070 2023-06-09T04:35:26.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.282+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a01e8530 con 0x7fc0a00b2070 2023-06-09T04:35:26.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.282+0000 7fc087fff700 1 -- 172.21.15.92:0/3370930498 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc094013c80 con 0x7fc0a00b2070 2023-06-09T04:35:26.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.283+0000 7fc087fff700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc08c06ace0 0x7fc08c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.283+0000 7fc087fff700 1 -- 172.21.15.92:0/3370930498 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc0940891e0 con 0x7fc0a00b2070 2023-06-09T04:35:26.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.283+0000 7fc09e7fc700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc08c06ace0 0x7fc08c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.283+0000 7fc09e7fc700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc08c06ace0 0x7fc08c06d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0a006cc40 tx=0x7fc090008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:26.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.287+0000 7fc087fff700 1 -- 172.21.15.92:0/3370930498 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc094054200 con 0x7fc0a00b2070 2023-06-09T04:35:26.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.470+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 --> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0a00d0fc0 con 0x7fc08c06ace0 2023-06-09T04:35:26.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.478+0000 7fc087fff700 1 -- 172.21.15.92:0/3370930498 <== mgr.4104 v2:172.21.15.92:6832/3112104013 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286634 (secure 0 0 0) 0x7fc09000bb40 con 0x7fc08c06ace0 2023-06-09T04:35:26.481 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:26.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc08c06ace0 msgr2=0x7fc08c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc08c06ace0 0x7fc08c06d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc0a006cc40 tx=0x7fc090008030 comp rx=0 tx=0).stop 2023-06-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00b2070 msgr2=0x7fc0a015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00b2070 0x7fc0a015a3a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc09400e840 tx=0x7fc09400ec00 comp rx=0 tx=0).stop 2023-06-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 shutdown_connections 2023-06-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc08c06ace0 0x7fc08c06d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0a00cfa70 0x7fc0a01e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a00b2070 0x7fc0a015a3a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 --2- 172.21.15.92:0/3370930498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0a00cd1a0 0x7fc0a01e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 >> 172.21.15.92:0/3370930498 conn(0x7fc0a0065a50 msgr2=0x7fc0a00d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:26.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 shutdown_connections 2023-06-09T04:35:26.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.481+0000 7fc0a5819700 1 -- 172.21.15.92:0/3370930498 wait complete. 2023-06-09T04:35:26.484 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-06-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-06-09T04:35:25.845950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218820,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755900,"statfs":{"total":773094113280,"available":772870041600,"internally_reserved":0,"allocated":3280896,"data_stored":1938951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.431818"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.956059+0000","last_change":"2023-06-09T04:35:17.427755+0000","last_active":"2023-06-09T04:35:18.956059+0000","last_peered":"2023-06-09T04:35:18.956059+0000","last_clean":"2023-06-09T04:35:18.956059+0000","last_became_active":"2023-06-09T04:35:17.427498+0000","last_became_peered":"2023-06-09T04:35:17.427498+0000","last_unstale":"2023-06-09T04:35:18.956059+0000","last_undegraded":"2023-06-09T04:35:18.956059+0000","last_fullsized":"2023-06-09T04:35:18.956059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:05.638304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423895+0000","last_change":"2023-06-09T04:35:17.438248+0000","last_active":"2023-06-09T04:35:18.423895+0000","last_peered":"2023-06-09T04:35:18.423895+0000","last_clean":"2023-06-09T04:35:18.423895+0000","last_became_active":"2023-06-09T04:35:17.437959+0000","last_became_peered":"2023-06-09T04:35:17.437959+0000","last_unstale":"2023-06-09T04:35:18.423895+0000","last_undegraded":"2023-06-09T04:35:18.423895+0000","last_fullsized":"2023-06-09T04:35:18.423895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071287+0000","last_change":"2023-06-09T04:35:17.444189+0000","last_active":"2023-06-09T04:35:19.071287+0000","last_peered":"2023-06-09T04:35:19.071287+0000","last_clean":"2023-06-09T04:35:19.071287+0000","last_became_active":"2023-06-09T04:35:17.443878+0000","last_became_peered":"2023-06-09T04:35:17.443878+0000","last_unstale":"2023-06-09T04:35:19.071287+0000","last_undegraded":"2023-06-09T04:35:19.071287+0000","last_fullsized":"2023-06-09T04:35:19.071287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:01.857396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423293+0000","last_change":"2023-06-09T04:35:17.439643+0000","last_active":"2023-06-09T04:35:18.423293+0000","last_peered":"2023-06-09T04:35:18.423293+0000","last_clean":"2023-06-09T04:35:18.423293+0000","last_became_active":"2023-06-09T04:35:17.439270+0000","last_became_peered":"2023-06-09T04:35:17.439270+0000","last_unstale":"2023-06-09T04:35:18.423293+0000","last_undegraded":"2023-06-09T04:35:18.423293+0000","last_fullsized":"2023-06-09T04:35:18.423293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:05.638304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353770+0000","last_change":"2023-06-09T04:35:17.440592+0000","last_active":"2023-06-09T04:35:20.353770+0000","last_peered":"2023-06-09T04:35:20.353770+0000","last_clean":"2023-06-09T04:35:20.353770+0000","last_became_active":"2023-06-09T04:35:17.440212+0000","last_became_peered":"2023-06-09T04:35:17.440212+0000","last_unstale":"2023-06-09T04:35:20.353770+0000","last_undegraded":"2023-06-09T04:35:20.353770+0000","last_fullsized":"2023-06-09T04:35:20.353770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:35:03.891489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070853+0000","last_change":"2023-06-09T04:35:17.439583+0000","last_active":"2023-06-09T04:35:19.070853+0000","last_peered":"2023-06-09T04:35:19.070853+0000","last_clean":"2023-06-09T04:35:19.070853+0000","last_became_active":"2023-06-09T04:35:17.439241+0000","last_became_peered":"2023-06-09T04:35:17.439241+0000","last_unstale":"2023-06-09T04:35:19.070853+0000","last_undegraded":"2023-06-09T04:35:19.070853+0000","last_fullsized":"2023-06-09T04:35:19.070853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243375+0000","last_change":"2023-06-09T04:35:17.437095+0000","last_active":"2023-06-09T04:35:20.243375+0000","last_peered":"2023-06-09T04:35:20.243375+0000","last_clean":"2023-06-09T04:35:20.243375+0000","last_became_active":"2023-06-09T04:35:17.436222+0000","last_became_peered":"2023-06-09T04:35:17.436222+0000","last_unstale":"2023-06-09T04:35:20.243375+0000","last_undegraded":"2023-06-09T04:35:20.243375+0000","last_fullsized":"2023-06-09T04:35:20.243375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:01.857396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423746+0000","last_change":"2023-06-09T04:35:17.439627+0000","last_active":"2023-06-09T04:35:18.423746+0000","last_peered":"2023-06-09T04:35:18.423746+0000","last_clean":"2023-06-09T04:35:18.423746+0000","last_became_active":"2023-06-09T04:35:17.439339+0000","last_became_peered":"2023-06-09T04:35:17.439339+0000","last_unstale":"2023-06-09T04:35:18.423746+0000","last_undegraded":"2023-06-09T04:35:18.423746+0000","last_fullsized":"2023-06-09T04:35:18.423746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353891+0000","last_change":"2023-06-09T04:35:17.434172+0000","last_active":"2023-06-09T04:35:20.353891+0000","last_peered":"2023-06-09T04:35:20.353891+0000","last_clean":"2023-06-09T04:35:20.353891+0000","last_became_active":"2023-06-09T04:35:17.433941+0000","last_became_peered":"2023-06-09T04:35:17.433941+0000","last_unstale":"2023-06-09T04:35:20.353891+0000","last_undegraded":"2023-06-09T04:35:20.353891+0000","last_fullsized":"2023-06-09T04:35:20.353891+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423982+0000","last_change":"2023-06-09T04:35:17.441671+0000","last_active":"2023-06-09T04:35:18.423982+0000","last_peered":"2023-06-09T04:35:18.423982+0000","last_clean":"2023-06-09T04:35:18.423982+0000","last_became_active":"2023-06-09T04:35:17.441387+0000","last_became_peered":"2023-06-09T04:35:17.441387+0000","last_unstale":"2023-06-09T04:35:18.423982+0000","last_undegraded":"2023-06-09T04:35:18.423982+0000","last_fullsized":"2023-06-09T04:35:18.423982+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071335+0000","last_change":"2023-06-09T04:35:17.438983+0000","last_active":"2023-06-09T04:35:19.071335+0000","last_peered":"2023-06-09T04:35:19.071335+0000","last_clean":"2023-06-09T04:35:19.071335+0000","last_became_active":"2023-06-09T04:35:17.438684+0000","last_became_peered":"2023-06-09T04:35:17.438684+0000","last_unstale":"2023-06-09T04:35:19.071335+0000","last_undegraded":"2023-06-09T04:35:19.071335+0000","last_fullsized":"2023-06-09T04:35:19.071335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:15.755801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955691+0000","last_change":"2023-06-09T04:35:17.433301+0000","last_active":"2023-06-09T04:35:18.955691+0000","last_peered":"2023-06-09T04:35:18.955691+0000","last_clean":"2023-06-09T04:35:18.955691+0000","last_became_active":"2023-06-09T04:35:17.433031+0000","last_became_peered":"2023-06-09T04:35:17.433031+0000","last_unstale":"2023-06-09T04:35:18.955691+0000","last_undegraded":"2023-06-09T04:35:18.955691+0000","last_fullsized":"2023-06-09T04:35:18.955691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955577+0000","last_change":"2023-06-09T04:35:17.432672+0000","last_active":"2023-06-09T04:35:18.955577+0000","last_peered":"2023-06-09T04:35:18.955577+0000","last_clean":"2023-06-09T04:35:18.955577+0000","last_became_active":"2023-06-09T04:35:17.432388+0000","last_became_peered":"2023-06-09T04:35:17.432388+0000","last_unstale":"2023-06-09T04:35:18.955577+0000","last_undegraded":"2023-06-09T04:35:18.955577+0000","last_fullsized":"2023-06-09T04:35:18.955577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424293+0000","last_change":"2023-06-09T04:35:17.439222+0000","last_active":"2023-06-09T04:35:18.424293+0000","last_peered":"2023-06-09T04:35:18.424293+0000","last_clean":"2023-06-09T04:35:18.424293+0000","last_became_active":"2023-06-09T04:35:17.438873+0000","last_became_peered":"2023-06-09T04:35:17.438873+0000","last_unstale":"2023-06-09T04:35:18.424293+0000","last_undegraded":"2023-06-09T04:35:18.424293+0000","last_fullsized":"2023-06-09T04:35:18.424293+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243743+0000","last_change":"2023-06-09T04:35:17.437077+0000","last_active":"2023-06-09T04:35:20.243743+0000","last_peered":"2023-06-09T04:35:20.243743+0000","last_clean":"2023-06-09T04:35:20.243743+0000","last_became_active":"2023-06-09T04:35:17.436197+0000","last_became_peered":"2023-06-09T04:35:17.436197+0000","last_unstale":"2023-06-09T04:35:20.243743+0000","last_undegraded":"2023-06-09T04:35:20.243743+0000","last_fullsized":"2023-06-09T04:35:20.243743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:53:57.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070817+0000","last_change":"2023-06-09T04:35:17.443763+0000","last_active":"2023-06-09T04:35:19.070817+0000","last_peered":"2023-06-09T04:35:19.070817+0000","last_clean":"2023-06-09T04:35:19.070817+0000","last_became_active":"2023-06-09T04:35:17.442440+0000","last_became_peered":"2023-06-09T04:35:17.442440+0000","last_unstale":"2023-06-09T04:35:19.070817+0000","last_undegraded":"2023-06-09T04:35:19.070817+0000","last_fullsized":"2023-06-09T04:35:19.070817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:55.606991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354004+0000","last_change":"2023-06-09T04:35:17.433713+0000","last_active":"2023-06-09T04:35:20.354004+0000","last_peered":"2023-06-09T04:35:20.354004+0000","last_clean":"2023-06-09T04:35:20.354004+0000","last_became_active":"2023-06-09T04:35:17.433428+0000","last_became_peered":"2023-06-09T04:35:17.433428+0000","last_unstale":"2023-06-09T04:35:20.354004+0000","last_undegraded":"2023-06-09T04:35:20.354004+0000","last_fullsized":"2023-06-09T04:35:20.354004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071439+0000","last_change":"2023-06-09T04:35:17.432392+0000","last_active":"2023-06-09T04:35:19.071439+0000","last_peered":"2023-06-09T04:35:19.071439+0000","last_clean":"2023-06-09T04:35:19.071439+0000","last_became_active":"2023-06-09T04:35:17.432126+0000","last_became_peered":"2023-06-09T04:35:17.432126+0000","last_unstale":"2023-06-09T04:35:19.071439+0000","last_undegraded":"2023-06-09T04:35:19.071439+0000","last_fullsized":"2023-06-09T04:35:19.071439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:58:27.739525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955661+0000","last_change":"2023-06-09T04:35:17.431541+0000","last_active":"2023-06-09T04:35:18.955661+0000","last_peered":"2023-06-09T04:35:18.955661+0000","last_clean":"2023-06-09T04:35:18.955661+0000","last_became_active":"2023-06-09T04:35:17.431276+0000","last_became_peered":"2023-06-09T04:35:17.431276+0000","last_unstale":"2023-06-09T04:35:18.955661+0000","last_undegraded":"2023-06-09T04:35:18.955661+0000","last_fullsized":"2023-06-09T04:35:18.955661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:02:00.587762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071083+0000","last_change":"2023-06-09T04:35:17.445238+0000","last_active":"2023-06-09T04:35:19.071083+0000","last_peered":"2023-06-09T04:35:19.071083+0000","last_clean":"2023-06-09T04:35:19.071083+0000","last_became_active":"2023-06-09T04:35:17.444919+0000","last_became_peered":"2023-06-09T04:35:17.444919+0000","last_unstale":"2023-06-09T04:35:19.071083+0000","last_undegraded":"2023-06-09T04:35:19.071083+0000","last_fullsized":"2023-06-09T04:35:19.071083+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:04.391938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243586+0000","last_change":"2023-06-09T04:35:17.441967+0000","last_active":"2023-06-09T04:35:20.243586+0000","last_peered":"2023-06-09T04:35:20.243586+0000","last_clean":"2023-06-09T04:35:20.243586+0000","last_became_active":"2023-06-09T04:35:17.441745+0000","last_became_peered":"2023-06-09T04:35:17.441745+0000","last_unstale":"2023-06-09T04:35:20.243586+0000","last_undegraded":"2023-06-09T04:35:20.243586+0000","last_fullsized":"2023-06-09T04:35:20.243586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:38.026198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354357+0000","last_change":"2023-06-09T04:35:17.440445+0000","last_active":"2023-06-09T04:35:20.354357+0000","last_peered":"2023-06-09T04:35:20.354357+0000","last_clean":"2023-06-09T04:35:20.354357+0000","last_became_active":"2023-06-09T04:35:17.440125+0000","last_became_peered":"2023-06-09T04:35:17.440125+0000","last_unstale":"2023-06-09T04:35:20.354357+0000","last_undegraded":"2023-06-09T04:35:20.354357+0000","last_fullsized":"2023-06-09T04:35:20.354357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423869+0000","last_change":"2023-06-09T04:35:17.442303+0000","last_active":"2023-06-09T04:35:18.423869+0000","last_peered":"2023-06-09T04:35:18.423869+0000","last_clean":"2023-06-09T04:35:18.423869+0000","last_became_active":"2023-06-09T04:35:17.442041+0000","last_became_peered":"2023-06-09T04:35:17.442041+0000","last_unstale":"2023-06-09T04:35:18.423869+0000","last_undegraded":"2023-06-09T04:35:18.423869+0000","last_fullsized":"2023-06-09T04:35:18.423869+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243381+0000","last_change":"2023-06-09T04:35:17.436865+0000","last_active":"2023-06-09T04:35:20.243381+0000","last_peered":"2023-06-09T04:35:20.243381+0000","last_clean":"2023-06-09T04:35:20.243381+0000","last_became_active":"2023-06-09T04:35:17.436545+0000","last_became_peered":"2023-06-09T04:35:17.436545+0000","last_unstale":"2023-06-09T04:35:20.243381+0000","last_undegraded":"2023-06-09T04:35:20.243381+0000","last_fullsized":"2023-06-09T04:35:20.243381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353900+0000","last_change":"2023-06-09T04:35:17.442515+0000","last_active":"2023-06-09T04:35:20.353900+0000","last_peered":"2023-06-09T04:35:20.353900+0000","last_clean":"2023-06-09T04:35:20.353900+0000","last_became_active":"2023-06-09T04:35:17.442247+0000","last_became_peered":"2023-06-09T04:35:17.442247+0000","last_unstale":"2023-06-09T04:35:20.353900+0000","last_undegraded":"2023-06-09T04:35:20.353900+0000","last_fullsized":"2023-06-09T04:35:20.353900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354958+0000","last_change":"2023-06-09T04:35:17.435870+0000","last_active":"2023-06-09T04:35:20.354958+0000","last_peered":"2023-06-09T04:35:20.354958+0000","last_clean":"2023-06-09T04:35:20.354958+0000","last_became_active":"2023-06-09T04:35:17.435616+0000","last_became_peered":"2023-06-09T04:35:17.435616+0000","last_unstale":"2023-06-09T04:35:20.354958+0000","last_undegraded":"2023-06-09T04:35:20.354958+0000","last_fullsized":"2023-06-09T04:35:20.354958+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:25.978819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424017+0000","last_change":"2023-06-09T04:35:17.438485+0000","last_active":"2023-06-09T04:35:18.424017+0000","last_peered":"2023-06-09T04:35:18.424017+0000","last_clean":"2023-06-09T04:35:18.424017+0000","last_became_active":"2023-06-09T04:35:17.438173+0000","last_became_peered":"2023-06-09T04:35:17.438173+0000","last_unstale":"2023-06-09T04:35:18.424017+0000","last_undegraded":"2023-06-09T04:35:18.424017+0000","last_fullsized":"2023-06-09T04:35:18.424017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:03.444952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071067+0000","last_change":"2023-06-09T04:35:17.032888+0000","last_active":"2023-06-09T04:35:19.071067+0000","last_peered":"2023-06-09T04:35:19.071067+0000","last_clean":"2023-06-09T04:35:19.071067+0000","last_became_active":"2023-06-09T04:35:17.032675+0000","last_became_peered":"2023-06-09T04:35:17.032675+0000","last_unstale":"2023-06-09T04:35:19.071067+0000","last_undegraded":"2023-06-09T04:35:19.071067+0000","last_fullsized":"2023-06-09T04:35:19.071067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:02.400588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424118+0000","last_change":"2023-06-09T04:35:17.439014+0000","last_active":"2023-06-09T04:35:18.424118+0000","last_peered":"2023-06-09T04:35:18.424118+0000","last_clean":"2023-06-09T04:35:18.424118+0000","last_became_active":"2023-06-09T04:35:17.438737+0000","last_became_peered":"2023-06-09T04:35:17.438737+0000","last_unstale":"2023-06-09T04:35:18.424118+0000","last_undegraded":"2023-06-09T04:35:18.424118+0000","last_fullsized":"2023-06-09T04:35:18.424118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:03.406729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353995+0000","last_change":"2023-06-09T04:35:17.029890+0000","last_active":"2023-06-09T04:35:20.353995+0000","last_peered":"2023-06-09T04:35:20.353995+0000","last_clean":"2023-06-09T04:35:20.353995+0000","last_became_active":"2023-06-09T04:35:17.029430+0000","last_became_peered":"2023-06-09T04:35:17.029430+0000","last_unstale":"2023-06-09T04:35:20.353995+0000","last_undegraded":"2023-06-09T04:35:20.353995+0000","last_fullsized":"2023-06-09T04:35:20.353995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:54.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354046+0000","last_change":"2023-06-09T04:35:17.430052+0000","last_active":"2023-06-09T04:35:20.354046+0000","last_peered":"2023-06-09T04:35:20.354046+0000","last_clean":"2023-06-09T04:35:20.354046+0000","last_became_active":"2023-06-09T04:35:17.429777+0000","last_became_peered":"2023-06-09T04:35:17.429777+0000","last_unstale":"2023-06-09T04:35:20.354046+0000","last_undegraded":"2023-06-09T04:35:20.354046+0000","last_fullsized":"2023-06-09T04:35:20.354046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423796+0000","last_change":"2023-06-09T04:35:17.030206+0000","last_active":"2023-06-09T04:35:18.423796+0000","last_peered":"2023-06-09T04:35:18.423796+0000","last_clean":"2023-06-09T04:35:18.423796+0000","last_became_active":"2023-06-09T04:35:17.029820+0000","last_became_peered":"2023-06-09T04:35:17.029820+0000","last_unstale":"2023-06-09T04:35:18.423796+0000","last_undegraded":"2023-06-09T04:35:18.423796+0000","last_fullsized":"2023-06-09T04:35:18.423796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423430+0000","last_change":"2023-06-09T04:35:17.440301+0000","last_active":"2023-06-09T04:35:18.423430+0000","last_peered":"2023-06-09T04:35:18.423430+0000","last_clean":"2023-06-09T04:35:18.423430+0000","last_became_active":"2023-06-09T04:35:17.440026+0000","last_became_peered":"2023-06-09T04:35:17.440026+0000","last_unstale":"2023-06-09T04:35:18.423430+0000","last_undegraded":"2023-06-09T04:35:18.423430+0000","last_fullsized":"2023-06-09T04:35:18.423430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:32:18.360318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354131+0000","last_change":"2023-06-09T04:35:17.029374+0000","last_active":"2023-06-09T04:35:20.354131+0000","last_peered":"2023-06-09T04:35:20.354131+0000","last_clean":"2023-06-09T04:35:20.354131+0000","last_became_active":"2023-06-09T04:35:17.029069+0000","last_became_peered":"2023-06-09T04:35:17.029069+0000","last_unstale":"2023-06-09T04:35:20.354131+0000","last_undegraded":"2023-06-09T04:35:20.354131+0000","last_fullsized":"2023-06-09T04:35:20.354131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423667+0000","last_change":"2023-06-09T04:35:17.436424+0000","last_active":"2023-06-09T04:35:18.423667+0000","last_peered":"2023-06-09T04:35:18.423667+0000","last_clean":"2023-06-09T04:35:18.423667+0000","last_became_active":"2023-06-09T04:35:17.436173+0000","last_became_peered":"2023-06-09T04:35:17.436173+0000","last_unstale":"2023-06-09T04:35:18.423667+0000","last_undegraded":"2023-06-09T04:35:18.423667+0000","last_fullsized":"2023-06-09T04:35:18.423667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:35:03.891489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423864+0000","last_change":"2023-06-09T04:35:17.024674+0000","last_active":"2023-06-09T04:35:18.423864+0000","last_peered":"2023-06-09T04:35:18.423864+0000","last_clean":"2023-06-09T04:35:18.423864+0000","last_became_active":"2023-06-09T04:35:17.024402+0000","last_became_peered":"2023-06-09T04:35:17.024402+0000","last_unstale":"2023-06-09T04:35:18.423864+0000","last_undegraded":"2023-06-09T04:35:18.423864+0000","last_fullsized":"2023-06-09T04:35:18.423864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423738+0000","last_change":"2023-06-09T04:35:17.431496+0000","last_active":"2023-06-09T04:35:18.423738+0000","last_peered":"2023-06-09T04:35:18.423738+0000","last_clean":"2023-06-09T04:35:18.423738+0000","last_became_active":"2023-06-09T04:35:17.431197+0000","last_became_peered":"2023-06-09T04:35:17.431197+0000","last_unstale":"2023-06-09T04:35:18.423738+0000","last_undegraded":"2023-06-09T04:35:18.423738+0000","last_fullsized":"2023-06-09T04:35:18.423738+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243596+0000","last_change":"2023-06-09T04:35:17.029451+0000","last_active":"2023-06-09T04:35:20.243596+0000","last_peered":"2023-06-09T04:35:20.243596+0000","last_clean":"2023-06-09T04:35:20.243596+0000","last_became_active":"2023-06-09T04:35:17.029169+0000","last_became_peered":"2023-06-09T04:35:17.029169+0000","last_unstale":"2023-06-09T04:35:20.243596+0000","last_undegraded":"2023-06-09T04:35:20.243596+0000","last_fullsized":"2023-06-09T04:35:20.243596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:41:14.749659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354582+0000","last_change":"2023-06-09T04:35:17.438193+0000","last_active":"2023-06-09T04:35:20.354582+0000","last_peered":"2023-06-09T04:35:20.354582+0000","last_clean":"2023-06-09T04:35:20.354582+0000","last_became_active":"2023-06-09T04:35:17.437765+0000","last_became_peered":"2023-06-09T04:35:17.437765+0000","last_unstale":"2023-06-09T04:35:20.354582+0000","last_undegraded":"2023-06-09T04:35:20.354582+0000","last_fullsized":"2023-06-09T04:35:20.354582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:03.444952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071305+0000","last_change":"2023-06-09T04:35:17.028442+0000","last_active":"2023-06-09T04:35:19.071305+0000","last_peered":"2023-06-09T04:35:19.071305+0000","last_clean":"2023-06-09T04:35:19.071305+0000","last_became_active":"2023-06-09T04:35:17.028162+0000","last_became_peered":"2023-06-09T04:35:17.028162+0000","last_unstale":"2023-06-09T04:35:19.071305+0000","last_undegraded":"2023-06-09T04:35:19.071305+0000","last_fullsized":"2023-06-09T04:35:19.071305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:37.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071228+0000","last_change":"2023-06-09T04:35:17.433199+0000","last_active":"2023-06-09T04:35:19.071228+0000","last_peered":"2023-06-09T04:35:19.071228+0000","last_clean":"2023-06-09T04:35:19.071228+0000","last_became_active":"2023-06-09T04:35:17.432807+0000","last_became_peered":"2023-06-09T04:35:17.432807+0000","last_unstale":"2023-06-09T04:35:19.071228+0000","last_undegraded":"2023-06-09T04:35:19.071228+0000","last_fullsized":"2023-06-09T04:35:19.071228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:56:00.781717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243460+0000","last_change":"2023-06-09T04:35:17.029495+0000","last_active":"2023-06-09T04:35:20.243460+0000","last_peered":"2023-06-09T04:35:20.243460+0000","last_clean":"2023-06-09T04:35:20.243460+0000","last_became_active":"2023-06-09T04:35:17.028510+0000","last_became_peered":"2023-06-09T04:35:17.028510+0000","last_unstale":"2023-06-09T04:35:20.243460+0000","last_undegraded":"2023-06-09T04:35:20.243460+0000","last_fullsized":"2023-06-09T04:35:20.243460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:24.972678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423724+0000","last_change":"2023-06-09T04:35:17.436310+0000","last_active":"2023-06-09T04:35:18.423724+0000","last_peered":"2023-06-09T04:35:18.423724+0000","last_clean":"2023-06-09T04:35:18.423724+0000","last_became_active":"2023-06-09T04:35:17.436034+0000","last_became_peered":"2023-06-09T04:35:17.436034+0000","last_unstale":"2023-06-09T04:35:18.423724+0000","last_undegraded":"2023-06-09T04:35:18.423724+0000","last_fullsized":"2023-06-09T04:35:18.423724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353670+0000","last_change":"2023-06-09T04:35:17.028971+0000","last_active":"2023-06-09T04:35:20.353670+0000","last_peered":"2023-06-09T04:35:20.353670+0000","last_clean":"2023-06-09T04:35:20.353670+0000","last_became_active":"2023-06-09T04:35:17.028680+0000","last_became_peered":"2023-06-09T04:35:17.028680+0000","last_unstale":"2023-06-09T04:35:20.353670+0000","last_undegraded":"2023-06-09T04:35:20.353670+0000","last_fullsized":"2023-06-09T04:35:20.353670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:29.160371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424134+0000","last_change":"2023-06-09T04:35:17.438269+0000","last_active":"2023-06-09T04:35:18.424134+0000","last_peered":"2023-06-09T04:35:18.424134+0000","last_clean":"2023-06-09T04:35:18.424134+0000","last_became_active":"2023-06-09T04:35:17.437854+0000","last_became_peered":"2023-06-09T04:35:17.437854+0000","last_unstale":"2023-06-09T04:35:18.424134+0000","last_undegraded":"2023-06-09T04:35:18.424134+0000","last_fullsized":"2023-06-09T04:35:18.424134+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243547+0000","last_change":"2023-06-09T04:35:17.028793+0000","last_active":"2023-06-09T04:35:20.243547+0000","last_peered":"2023-06-09T04:35:20.243547+0000","last_clean":"2023-06-09T04:35:20.243547+0000","last_became_active":"2023-06-09T04:35:17.028480+0000","last_became_peered":"2023-06-09T04:35:17.028480+0000","last_unstale":"2023-06-09T04:35:20.243547+0000","last_undegraded":"2023-06-09T04:35:20.243547+0000","last_fullsized":"2023-06-09T04:35:20.243547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:59.775576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354124+0000","last_change":"2023-06-09T04:35:17.427519+0000","last_active":"2023-06-09T04:35:20.354124+0000","last_peered":"2023-06-09T04:35:20.354124+0000","last_clean":"2023-06-09T04:35:20.354124+0000","last_became_active":"2023-06-09T04:35:17.427246+0000","last_became_peered":"2023-06-09T04:35:17.427246+0000","last_unstale":"2023-06-09T04:35:20.354124+0000","last_undegraded":"2023-06-09T04:35:20.354124+0000","last_fullsized":"2023-06-09T04:35:20.354124+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423219+0000","last_change":"2023-06-09T04:35:17.030556+0000","last_active":"2023-06-09T04:35:18.423219+0000","last_peered":"2023-06-09T04:35:18.423219+0000","last_clean":"2023-06-09T04:35:18.423219+0000","last_became_active":"2023-06-09T04:35:17.030279+0000","last_became_peered":"2023-06-09T04:35:17.030279+0000","last_unstale":"2023-06-09T04:35:18.423219+0000","last_undegraded":"2023-06-09T04:35:18.423219+0000","last_fullsized":"2023-06-09T04:35:18.423219+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:01:59.581621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424220+0000","last_change":"2023-06-09T04:35:17.431297+0000","last_active":"2023-06-09T04:35:18.424220+0000","last_peered":"2023-06-09T04:35:18.424220+0000","last_clean":"2023-06-09T04:35:18.424220+0000","last_became_active":"2023-06-09T04:35:17.431025+0000","last_became_peered":"2023-06-09T04:35:17.431025+0000","last_unstale":"2023-06-09T04:35:18.424220+0000","last_undegraded":"2023-06-09T04:35:18.424220+0000","last_fullsized":"2023-06-09T04:35:18.424220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243918+0000","last_change":"2023-06-09T04:35:17.028833+0000","last_active":"2023-06-09T04:35:20.243918+0000","last_peered":"2023-06-09T04:35:20.243918+0000","last_clean":"2023-06-09T04:35:20.243918+0000","last_became_active":"2023-06-09T04:35:17.028375+0000","last_became_peered":"2023-06-09T04:35:17.028375+0000","last_unstale":"2023-06-09T04:35:20.243918+0000","last_undegraded":"2023-06-09T04:35:20.243918+0000","last_fullsized":"2023-06-09T04:35:20.243918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:44:50.086544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423784+0000","last_change":"2023-06-09T04:35:17.442725+0000","last_active":"2023-06-09T04:35:18.423784+0000","last_peered":"2023-06-09T04:35:18.423784+0000","last_clean":"2023-06-09T04:35:18.423784+0000","last_became_active":"2023-06-09T04:35:17.442455+0000","last_became_peered":"2023-06-09T04:35:17.442455+0000","last_unstale":"2023-06-09T04:35:18.423784+0000","last_undegraded":"2023-06-09T04:35:18.423784+0000","last_fullsized":"2023-06-09T04:35:18.423784+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243859+0000","last_change":"2023-06-09T04:35:17.029278+0000","last_active":"2023-06-09T04:35:20.243859+0000","last_peered":"2023-06-09T04:35:20.243859+0000","last_clean":"2023-06-09T04:35:20.243859+0000","last_became_active":"2023-06-09T04:35:17.028978+0000","last_became_peered":"2023-06-09T04:35:17.028978+0000","last_unstale":"2023-06-09T04:35:20.243859+0000","last_undegraded":"2023-06-09T04:35:20.243859+0000","last_fullsized":"2023-06-09T04:35:20.243859+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:17:16.655161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070861+0000","last_change":"2023-06-09T04:35:17.438515+0000","last_active":"2023-06-09T04:35:19.070861+0000","last_peered":"2023-06-09T04:35:19.070861+0000","last_clean":"2023-06-09T04:35:19.070861+0000","last_became_active":"2023-06-09T04:35:17.437901+0000","last_became_peered":"2023-06-09T04:35:17.437901+0000","last_unstale":"2023-06-09T04:35:19.070861+0000","last_undegraded":"2023-06-09T04:35:19.070861+0000","last_fullsized":"2023-06-09T04:35:19.070861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243775+0000","last_change":"2023-06-09T04:35:17.028112+0000","last_active":"2023-06-09T04:35:20.243775+0000","last_peered":"2023-06-09T04:35:20.243775+0000","last_clean":"2023-06-09T04:35:20.243775+0000","last_became_active":"2023-06-09T04:35:17.027836+0000","last_became_peered":"2023-06-09T04:35:17.027836+0000","last_unstale":"2023-06-09T04:35:20.243775+0000","last_undegraded":"2023-06-09T04:35:20.243775+0000","last_fullsized":"2023-06-09T04:35:20.243775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:25:10.314437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423291+0000","last_change":"2023-06-09T04:35:17.436148+0000","last_active":"2023-06-09T04:35:18.423291+0000","last_peered":"2023-06-09T04:35:18.423291+0000","last_clean":"2023-06-09T04:35:18.423291+0000","last_became_active":"2023-06-09T04:35:17.435824+0000","last_became_peered":"2023-06-09T04:35:17.435824+0000","last_unstale":"2023-06-09T04:35:18.423291+0000","last_undegraded":"2023-06-09T04:35:18.423291+0000","last_fullsized":"2023-06-09T04:35:18.423291+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:10.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.425087+0000","last_change":"2023-06-09T04:35:17.026454+0000","last_active":"2023-06-09T04:35:18.425087+0000","last_peered":"2023-06-09T04:35:18.425087+0000","last_clean":"2023-06-09T04:35:18.425087+0000","last_became_active":"2023-06-09T04:35:17.025128+0000","last_became_peered":"2023-06-09T04:35:17.025128+0000","last_unstale":"2023-06-09T04:35:18.425087+0000","last_undegraded":"2023-06-09T04:35:18.425087+0000","last_fullsized":"2023-06-09T04:35:18.425087+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:37.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354445+0000","last_change":"2023-06-09T04:35:17.436924+0000","last_active":"2023-06-09T04:35:20.354445+0000","last_peered":"2023-06-09T04:35:20.354445+0000","last_clean":"2023-06-09T04:35:20.354445+0000","last_became_active":"2023-06-09T04:35:17.436657+0000","last_became_peered":"2023-06-09T04:35:17.436657+0000","last_unstale":"2023-06-09T04:35:20.354445+0000","last_undegraded":"2023-06-09T04:35:20.354445+0000","last_fullsized":"2023-06-09T04:35:20.354445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:03.406729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423986+0000","last_change":"2023-06-09T04:35:17.031120+0000","last_active":"2023-06-09T04:35:18.423986+0000","last_peered":"2023-06-09T04:35:18.423986+0000","last_clean":"2023-06-09T04:35:18.423986+0000","last_became_active":"2023-06-09T04:35:17.030888+0000","last_became_peered":"2023-06-09T04:35:17.030888+0000","last_unstale":"2023-06-09T04:35:18.423986+0000","last_undegraded":"2023-06-09T04:35:18.423986+0000","last_fullsized":"2023-06-09T04:35:18.423986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:05.224171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071228+0000","last_change":"2023-06-09T04:35:17.432702+0000","last_active":"2023-06-09T04:35:19.071228+0000","last_peered":"2023-06-09T04:35:19.071228+0000","last_clean":"2023-06-09T04:35:19.071228+0000","last_became_active":"2023-06-09T04:35:17.432343+0000","last_became_peered":"2023-06-09T04:35:17.432343+0000","last_unstale":"2023-06-09T04:35:19.071228+0000","last_undegraded":"2023-06-09T04:35:19.071228+0000","last_fullsized":"2023-06-09T04:35:19.071228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:25.978819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423112+0000","last_change":"2023-06-09T04:35:17.028382+0000","last_active":"2023-06-09T04:35:18.423112+0000","last_peered":"2023-06-09T04:35:18.423112+0000","last_clean":"2023-06-09T04:35:18.423112+0000","last_became_active":"2023-06-09T04:35:17.028097+0000","last_became_peered":"2023-06-09T04:35:17.028097+0000","last_unstale":"2023-06-09T04:35:18.423112+0000","last_undegraded":"2023-06-09T04:35:18.423112+0000","last_fullsized":"2023-06-09T04:35:18.423112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:57.406268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071469+0000","last_change":"2023-06-09T04:35:17.434946+0000","last_active":"2023-06-09T04:35:19.071469+0000","last_peered":"2023-06-09T04:35:19.071469+0000","last_clean":"2023-06-09T04:35:19.071469+0000","last_became_active":"2023-06-09T04:35:17.434618+0000","last_became_peered":"2023-06-09T04:35:17.434618+0000","last_unstale":"2023-06-09T04:35:19.071469+0000","last_undegraded":"2023-06-09T04:35:19.071469+0000","last_fullsized":"2023-06-09T04:35:19.071469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:07:19.431879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071066+0000","last_change":"2023-06-09T04:35:17.023550+0000","last_active":"2023-06-09T04:35:19.071066+0000","last_peered":"2023-06-09T04:35:19.071066+0000","last_clean":"2023-06-09T04:35:19.071066+0000","last_became_active":"2023-06-09T04:35:17.022566+0000","last_became_peered":"2023-06-09T04:35:17.022566+0000","last_unstale":"2023-06-09T04:35:19.071066+0000","last_undegraded":"2023-06-09T04:35:19.071066+0000","last_fullsized":"2023-06-09T04:35:19.071066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071502+0000","last_change":"2023-06-09T04:35:17.431463+0000","last_active":"2023-06-09T04:35:19.071502+0000","last_peered":"2023-06-09T04:35:19.071502+0000","last_clean":"2023-06-09T04:35:19.071502+0000","last_became_active":"2023-06-09T04:35:17.431065+0000","last_became_peered":"2023-06-09T04:35:17.431065+0000","last_unstale":"2023-06-09T04:35:19.071502+0000","last_undegraded":"2023-06-09T04:35:19.071502+0000","last_fullsized":"2023-06-09T04:35:19.071502+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:53:57.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423829+0000","last_change":"2023-06-09T04:35:17.032982+0000","last_active":"2023-06-09T04:35:18.423829+0000","last_peered":"2023-06-09T04:35:18.423829+0000","last_clean":"2023-06-09T04:35:18.423829+0000","last_became_active":"2023-06-09T04:35:17.032679+0000","last_became_peered":"2023-06-09T04:35:17.032679+0000","last_unstale":"2023-06-09T04:35:18.423829+0000","last_undegraded":"2023-06-09T04:35:18.423829+0000","last_fullsized":"2023-06-09T04:35:18.423829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:54.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354242+0000","last_change":"2023-06-09T04:35:17.442076+0000","last_active":"2023-06-09T04:35:20.354242+0000","last_peered":"2023-06-09T04:35:20.354242+0000","last_clean":"2023-06-09T04:35:20.354242+0000","last_became_active":"2023-06-09T04:35:17.441806+0000","last_became_peered":"2023-06-09T04:35:17.441806+0000","last_unstale":"2023-06-09T04:35:20.354242+0000","last_undegraded":"2023-06-09T04:35:20.354242+0000","last_fullsized":"2023-06-09T04:35:20.354242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:40:01.838811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354557+0000","last_change":"2023-06-09T04:35:12.072943+0000","last_active":"2023-06-09T04:35:20.354557+0000","last_peered":"2023-06-09T04:35:20.354557+0000","last_clean":"2023-06-09T04:35:20.354557+0000","last_became_active":"2023-06-09T04:35:12.072623+0000","last_became_peered":"2023-06-09T04:35:12.072623+0000","last_unstale":"2023-06-09T04:35:20.354557+0000","last_undegraded":"2023-06-09T04:35:20.354557+0000","last_fullsized":"2023-06-09T04:35:20.354557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:00.202928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424301+0000","last_change":"2023-06-09T04:35:17.026144+0000","last_active":"2023-06-09T04:35:18.424301+0000","last_peered":"2023-06-09T04:35:18.424301+0000","last_clean":"2023-06-09T04:35:18.424301+0000","last_became_active":"2023-06-09T04:35:17.025699+0000","last_became_peered":"2023-06-09T04:35:17.025699+0000","last_unstale":"2023-06-09T04:35:18.424301+0000","last_undegraded":"2023-06-09T04:35:18.424301+0000","last_fullsized":"2023-06-09T04:35:18.424301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354751+0000","last_change":"2023-06-09T04:35:17.440088+0000","last_active":"2023-06-09T04:35:20.354751+0000","last_peered":"2023-06-09T04:35:20.354751+0000","last_clean":"2023-06-09T04:35:20.354751+0000","last_became_active":"2023-06-09T04:35:17.439771+0000","last_became_peered":"2023-06-09T04:35:17.439771+0000","last_unstale":"2023-06-09T04:35:20.354751+0000","last_undegraded":"2023-06-09T04:35:20.354751+0000","last_fullsized":"2023-06-09T04:35:20.354751+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424339+0000","last_change":"2023-06-09T04:35:12.002754+0000","last_active":"2023-06-09T04:35:18.424339+0000","last_peered":"2023-06-09T04:35:18.424339+0000","last_clean":"2023-06-09T04:35:18.424339+0000","last_became_active":"2023-06-09T04:35:12.002471+0000","last_became_peered":"2023-06-09T04:35:12.002471+0000","last_unstale":"2023-06-09T04:35:18.424339+0000","last_undegraded":"2023-06-09T04:35:18.424339+0000","last_fullsized":"2023-06-09T04:35:18.424339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:54.754333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071063+0000","last_change":"2023-06-09T04:35:17.032034+0000","last_active":"2023-06-09T04:35:19.071063+0000","last_peered":"2023-06-09T04:35:19.071063+0000","last_clean":"2023-06-09T04:35:19.071063+0000","last_became_active":"2023-06-09T04:35:17.031771+0000","last_became_peered":"2023-06-09T04:35:17.031771+0000","last_unstale":"2023-06-09T04:35:19.071063+0000","last_undegraded":"2023-06-09T04:35:19.071063+0000","last_fullsized":"2023-06-09T04:35:19.071063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:53.206518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354666+0000","last_change":"2023-06-09T04:35:17.439779+0000","last_active":"2023-06-09T04:35:20.354666+0000","last_peered":"2023-06-09T04:35:20.354666+0000","last_clean":"2023-06-09T04:35:20.354666+0000","last_became_active":"2023-06-09T04:35:17.439539+0000","last_became_peered":"2023-06-09T04:35:17.439539+0000","last_unstale":"2023-06-09T04:35:20.354666+0000","last_undegraded":"2023-06-09T04:35:20.354666+0000","last_fullsized":"2023-06-09T04:35:20.354666+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:41:03.406729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955491+0000","last_change":"2023-06-09T04:35:12.071356+0000","last_active":"2023-06-09T04:35:18.955491+0000","last_peered":"2023-06-09T04:35:18.955491+0000","last_clean":"2023-06-09T04:35:18.955491+0000","last_became_active":"2023-06-09T04:35:12.070997+0000","last_became_peered":"2023-06-09T04:35:12.070997+0000","last_unstale":"2023-06-09T04:35:18.955491+0000","last_undegraded":"2023-06-09T04:35:18.955491+0000","last_fullsized":"2023-06-09T04:35:18.955491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:20:52.385025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354331+0000","last_change":"2023-06-09T04:35:17.027888+0000","last_active":"2023-06-09T04:35:20.354331+0000","last_peered":"2023-06-09T04:35:20.354331+0000","last_clean":"2023-06-09T04:35:20.354331+0000","last_became_active":"2023-06-09T04:35:17.027381+0000","last_became_peered":"2023-06-09T04:35:17.027381+0000","last_unstale":"2023-06-09T04:35:20.354331+0000","last_undegraded":"2023-06-09T04:35:20.354331+0000","last_fullsized":"2023-06-09T04:35:20.354331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354038+0000","last_change":"2023-06-09T04:35:17.444159+0000","last_active":"2023-06-09T04:35:20.354038+0000","last_peered":"2023-06-09T04:35:20.354038+0000","last_clean":"2023-06-09T04:35:20.354038+0000","last_became_active":"2023-06-09T04:35:17.443862+0000","last_became_peered":"2023-06-09T04:35:17.443862+0000","last_unstale":"2023-06-09T04:35:20.354038+0000","last_undegraded":"2023-06-09T04:35:20.354038+0000","last_fullsized":"2023-06-09T04:35:20.354038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424131+0000","last_change":"2023-06-09T04:35:11.999027+0000","last_active":"2023-06-09T04:35:18.424131+0000","last_peered":"2023-06-09T04:35:18.424131+0000","last_clean":"2023-06-09T04:35:18.424131+0000","last_became_active":"2023-06-09T04:35:11.998723+0000","last_became_peered":"2023-06-09T04:35:11.998723+0000","last_unstale":"2023-06-09T04:35:18.424131+0000","last_undegraded":"2023-06-09T04:35:18.424131+0000","last_fullsized":"2023-06-09T04:35:18.424131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:40:57.379345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243437+0000","last_change":"2023-06-09T04:35:06.996716+0000","last_active":"2023-06-09T04:35:20.243437+0000","last_peered":"2023-06-09T04:35:20.243437+0000","last_clean":"2023-06-09T04:35:20.243437+0000","last_became_active":"2023-06-09T04:35:06.995565+0000","last_became_peered":"2023-06-09T04:35:06.995565+0000","last_unstale":"2023-06-09T04:35:20.243437+0000","last_undegraded":"2023-06-09T04:35:20.243437+0000","last_fullsized":"2023-06-09T04:35:20.243437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_clean_scrub_stamp":"2023-06-09T04:35:05.906836+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:23:53.292065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423735+0000","last_change":"2023-06-09T04:35:17.019063+0000","last_active":"2023-06-09T04:35:18.423735+0000","last_peered":"2023-06-09T04:35:18.423735+0000","last_clean":"2023-06-09T04:35:18.423735+0000","last_became_active":"2023-06-09T04:35:17.018762+0000","last_became_peered":"2023-06-09T04:35:17.018762+0000","last_unstale":"2023-06-09T04:35:18.423735+0000","last_undegraded":"2023-06-09T04:35:18.423735+0000","last_fullsized":"2023-06-09T04:35:18.423735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:29.160371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423249+0000","last_change":"2023-06-09T04:35:17.445339+0000","last_active":"2023-06-09T04:35:18.423249+0000","last_peered":"2023-06-09T04:35:18.423249+0000","last_clean":"2023-06-09T04:35:18.423249+0000","last_became_active":"2023-06-09T04:35:17.445113+0000","last_became_peered":"2023-06-09T04:35:17.445113+0000","last_unstale":"2023-06-09T04:35:18.423249+0000","last_undegraded":"2023-06-09T04:35:18.423249+0000","last_fullsized":"2023-06-09T04:35:18.423249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:45:15.236132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243611+0000","last_change":"2023-06-09T04:35:12.002654+0000","last_active":"2023-06-09T04:35:20.243611+0000","last_peered":"2023-06-09T04:35:20.243611+0000","last_clean":"2023-06-09T04:35:20.243611+0000","last_became_active":"2023-06-09T04:35:12.002320+0000","last_became_peered":"2023-06-09T04:35:12.002320+0000","last_unstale":"2023-06-09T04:35:20.243611+0000","last_undegraded":"2023-06-09T04:35:20.243611+0000","last_fullsized":"2023-06-09T04:35:20.243611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:58:21.712140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353644+0000","last_change":"2023-06-09T04:35:17.029537+0000","last_active":"2023-06-09T04:35:20.353644+0000","last_peered":"2023-06-09T04:35:20.353644+0000","last_clean":"2023-06-09T04:35:20.353644+0000","last_became_active":"2023-06-09T04:35:17.029185+0000","last_became_peered":"2023-06-09T04:35:17.029185+0000","last_unstale":"2023-06-09T04:35:20.353644+0000","last_undegraded":"2023-06-09T04:35:20.353644+0000","last_fullsized":"2023-06-09T04:35:20.353644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:38:43.929108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.353900+0000","last_change":"2023-06-09T04:35:17.435253+0000","last_active":"2023-06-09T04:35:20.353900+0000","last_peered":"2023-06-09T04:35:20.353900+0000","last_clean":"2023-06-09T04:35:20.353900+0000","last_became_active":"2023-06-09T04:35:17.434964+0000","last_became_peered":"2023-06-09T04:35:17.434964+0000","last_unstale":"2023-06-09T04:35:20.353900+0000","last_undegraded":"2023-06-09T04:35:20.353900+0000","last_fullsized":"2023-06-09T04:35:20.353900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:34:55.606991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.243708+0000","last_change":"2023-06-09T04:35:12.071967+0000","last_active":"2023-06-09T04:35:20.243708+0000","last_peered":"2023-06-09T04:35:20.243708+0000","last_clean":"2023-06-09T04:35:20.243708+0000","last_became_active":"2023-06-09T04:35:12.071565+0000","last_became_peered":"2023-06-09T04:35:12.071565+0000","last_unstale":"2023-06-09T04:35:20.243708+0000","last_undegraded":"2023-06-09T04:35:20.243708+0000","last_fullsized":"2023-06-09T04:35:20.243708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:07:13.404494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354701+0000","last_change":"2023-06-09T04:35:17.027679+0000","last_active":"2023-06-09T04:35:20.354701+0000","last_peered":"2023-06-09T04:35:20.354701+0000","last_clean":"2023-06-09T04:35:20.354701+0000","last_became_active":"2023-06-09T04:35:17.027357+0000","last_became_peered":"2023-06-09T04:35:17.027357+0000","last_unstale":"2023-06-09T04:35:20.354701+0000","last_undegraded":"2023-06-09T04:35:20.354701+0000","last_fullsized":"2023-06-09T04:35:20.354701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354195+0000","last_change":"2023-06-09T04:35:17.440902+0000","last_active":"2023-06-09T04:35:20.354195+0000","last_peered":"2023-06-09T04:35:20.354195+0000","last_clean":"2023-06-09T04:35:20.354195+0000","last_became_active":"2023-06-09T04:35:17.440622+0000","last_became_peered":"2023-06-09T04:35:17.440622+0000","last_unstale":"2023-06-09T04:35:20.354195+0000","last_undegraded":"2023-06-09T04:35:20.354195+0000","last_fullsized":"2023-06-09T04:35:20.354195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424216+0000","last_change":"2023-06-09T04:35:17.026812+0000","last_active":"2023-06-09T04:35:18.424216+0000","last_peered":"2023-06-09T04:35:18.424216+0000","last_clean":"2023-06-09T04:35:18.424216+0000","last_became_active":"2023-06-09T04:35:17.026394+0000","last_became_peered":"2023-06-09T04:35:17.026394+0000","last_unstale":"2023-06-09T04:35:18.424216+0000","last_undegraded":"2023-06-09T04:35:18.424216+0000","last_fullsized":"2023-06-09T04:35:18.424216+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:24:03.385797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354610+0000","last_change":"2023-06-09T04:35:12.003438+0000","last_active":"2023-06-09T04:35:20.354610+0000","last_peered":"2023-06-09T04:35:20.354610+0000","last_clean":"2023-06-09T04:35:20.354610+0000","last_became_active":"2023-06-09T04:35:12.003155+0000","last_became_peered":"2023-06-09T04:35:12.003155+0000","last_unstale":"2023-06-09T04:35:20.354610+0000","last_undegraded":"2023-06-09T04:35:20.354610+0000","last_fullsized":"2023-06-09T04:35:20.354610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:48.185275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423524+0000","last_change":"2023-06-09T04:35:17.443931+0000","last_active":"2023-06-09T04:35:18.423524+0000","last_peered":"2023-06-09T04:35:18.423524+0000","last_clean":"2023-06-09T04:35:18.423524+0000","last_became_active":"2023-06-09T04:35:17.443663+0000","last_became_peered":"2023-06-09T04:35:17.443663+0000","last_unstale":"2023-06-09T04:35:18.423524+0000","last_undegraded":"2023-06-09T04:35:18.423524+0000","last_fullsized":"2023-06-09T04:35:18.423524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:43:45.048336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423304+0000","last_change":"2023-06-09T04:35:12.003198+0000","last_active":"2023-06-09T04:35:18.423304+0000","last_peered":"2023-06-09T04:35:18.423304+0000","last_clean":"2023-06-09T04:35:18.423304+0000","last_became_active":"2023-06-09T04:35:12.002905+0000","last_became_peered":"2023-06-09T04:35:12.002905+0000","last_unstale":"2023-06-09T04:35:18.423304+0000","last_undegraded":"2023-06-09T04:35:18.423304+0000","last_fullsized":"2023-06-09T04:35:18.423304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:27:49.237891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354416+0000","last_change":"2023-06-09T04:35:17.033086+0000","last_active":"2023-06-09T04:35:20.354416+0000","last_peered":"2023-06-09T04:35:20.354416+0000","last_clean":"2023-06-09T04:35:20.354416+0000","last_became_active":"2023-06-09T04:35:17.032814+0000","last_became_peered":"2023-06-09T04:35:17.032814+0000","last_unstale":"2023-06-09T04:35:20.354416+0000","last_undegraded":"2023-06-09T04:35:20.354416+0000","last_fullsized":"2023-06-09T04:35:20.354416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:57.751951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070930+0000","last_change":"2023-06-09T04:35:17.438683+0000","last_active":"2023-06-09T04:35:19.070930+0000","last_peered":"2023-06-09T04:35:19.070930+0000","last_clean":"2023-06-09T04:35:19.070930+0000","last_became_active":"2023-06-09T04:35:17.438261+0000","last_became_peered":"2023-06-09T04:35:17.438261+0000","last_unstale":"2023-06-09T04:35:19.070930+0000","last_undegraded":"2023-06-09T04:35:19.070930+0000","last_fullsized":"2023-06-09T04:35:19.070930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:41:58.758092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354316+0000","last_change":"2023-06-09T04:35:17.031891+0000","last_active":"2023-06-09T04:35:20.354316+0000","last_peered":"2023-06-09T04:35:20.354316+0000","last_clean":"2023-06-09T04:35:20.354316+0000","last_became_active":"2023-06-09T04:35:17.031613+0000","last_became_peered":"2023-06-09T04:35:17.031613+0000","last_unstale":"2023-06-09T04:35:20.354316+0000","last_undegraded":"2023-06-09T04:35:20.354316+0000","last_fullsized":"2023-06-09T04:35:20.354316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.955690+0000","last_change":"2023-06-09T04:35:17.437403+0000","last_active":"2023-06-09T04:35:18.955690+0000","last_peered":"2023-06-09T04:35:18.955690+0000","last_clean":"2023-06-09T04:35:18.955690+0000","last_became_active":"2023-06-09T04:35:17.437106+0000","last_became_peered":"2023-06-09T04:35:17.437106+0000","last_unstale":"2023-06-09T04:35:18.955690+0000","last_undegraded":"2023-06-09T04:35:18.955690+0000","last_fullsized":"2023-06-09T04:35:18.955690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:10.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424990+0000","last_change":"2023-06-09T04:35:17.025288+0000","last_active":"2023-06-09T04:35:18.424990+0000","last_peered":"2023-06-09T04:35:18.424990+0000","last_clean":"2023-06-09T04:35:18.424990+0000","last_became_active":"2023-06-09T04:35:17.024862+0000","last_became_peered":"2023-06-09T04:35:17.024862+0000","last_unstale":"2023-06-09T04:35:18.424990+0000","last_undegraded":"2023-06-09T04:35:18.424990+0000","last_fullsized":"2023-06-09T04:35:18.424990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:00.851254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070993+0000","last_change":"2023-06-09T04:35:17.434826+0000","last_active":"2023-06-09T04:35:19.070993+0000","last_peered":"2023-06-09T04:35:19.070993+0000","last_clean":"2023-06-09T04:35:19.070993+0000","last_became_active":"2023-06-09T04:35:17.433157+0000","last_became_peered":"2023-06-09T04:35:17.433157+0000","last_unstale":"2023-06-09T04:35:19.070993+0000","last_undegraded":"2023-06-09T04:35:19.070993+0000","last_fullsized":"2023-06-09T04:35:19.070993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:06.230312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424370+0000","last_change":"2023-06-09T04:35:17.025975+0000","last_active":"2023-06-09T04:35:18.424370+0000","last_peered":"2023-06-09T04:35:18.424370+0000","last_clean":"2023-06-09T04:35:18.424370+0000","last_became_active":"2023-06-09T04:35:17.025358+0000","last_became_peered":"2023-06-09T04:35:17.025358+0000","last_unstale":"2023-06-09T04:35:18.424370+0000","last_undegraded":"2023-06-09T04:35:18.424370+0000","last_fullsized":"2023-06-09T04:35:18.424370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:24.972678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.071194+0000","last_change":"2023-06-09T04:35:17.435124+0000","last_active":"2023-06-09T04:35:19.071194+0000","last_peered":"2023-06-09T04:35:19.071194+0000","last_clean":"2023-06-09T04:35:19.071194+0000","last_became_active":"2023-06-09T04:35:17.434182+0000","last_became_peered":"2023-06-09T04:35:17.434182+0000","last_unstale":"2023-06-09T04:35:19.071194+0000","last_undegraded":"2023-06-09T04:35:19.071194+0000","last_fullsized":"2023-06-09T04:35:19.071194+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:35.496407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424031+0000","last_change":"2023-06-09T04:35:17.025538+0000","last_active":"2023-06-09T04:35:18.424031+0000","last_peered":"2023-06-09T04:35:18.424031+0000","last_clean":"2023-06-09T04:35:18.424031+0000","last_became_active":"2023-06-09T04:35:17.025170+0000","last_became_peered":"2023-06-09T04:35:17.025170+0000","last_unstale":"2023-06-09T04:35:18.424031+0000","last_undegraded":"2023-06-09T04:35:18.424031+0000","last_fullsized":"2023-06-09T04:35:18.424031+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:27:02.438811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423757+0000","last_change":"2023-06-09T04:35:17.437701+0000","last_active":"2023-06-09T04:35:18.423757+0000","last_peered":"2023-06-09T04:35:18.423757+0000","last_clean":"2023-06-09T04:35:18.423757+0000","last_became_active":"2023-06-09T04:35:17.437342+0000","last_became_peered":"2023-06-09T04:35:17.437342+0000","last_unstale":"2023-06-09T04:35:18.423757+0000","last_undegraded":"2023-06-09T04:35:18.423757+0000","last_fullsized":"2023-06-09T04:35:18.423757+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:17.418216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354858+0000","last_change":"2023-06-09T04:35:17.029009+0000","last_active":"2023-06-09T04:35:20.354858+0000","last_peered":"2023-06-09T04:35:20.354858+0000","last_clean":"2023-06-09T04:35:20.354858+0000","last_became_active":"2023-06-09T04:35:17.028662+0000","last_became_peered":"2023-06-09T04:35:17.028662+0000","last_unstale":"2023-06-09T04:35:20.354858+0000","last_undegraded":"2023-06-09T04:35:20.354858+0000","last_fullsized":"2023-06-09T04:35:20.354858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:16.000567+0000","last_clean_scrub_stamp":"2023-06-09T04:35:16.000567+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:14:34.490265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070711+0000","last_change":"2023-06-09T04:35:17.434890+0000","last_active":"2023-06-09T04:35:19.070711+0000","last_peered":"2023-06-09T04:35:19.070711+0000","last_clean":"2023-06-09T04:35:19.070711+0000","last_became_active":"2023-06-09T04:35:17.434589+0000","last_became_peered":"2023-06-09T04:35:17.434589+0000","last_unstale":"2023-06-09T04:35:19.070711+0000","last_undegraded":"2023-06-09T04:35:19.070711+0000","last_fullsized":"2023-06-09T04:35:19.070711+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:05:13.486687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423112+0000","last_change":"2023-06-09T04:35:17.431610+0000","last_active":"2023-06-09T04:35:18.423112+0000","last_peered":"2023-06-09T04:35:18.423112+0000","last_clean":"2023-06-09T04:35:18.423112+0000","last_became_active":"2023-06-09T04:35:17.431333+0000","last_became_peered":"2023-06-09T04:35:17.431333+0000","last_unstale":"2023-06-09T04:35:18.423112+0000","last_undegraded":"2023-06-09T04:35:18.423112+0000","last_fullsized":"2023-06-09T04:35:18.423112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:47:58.669043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.423602+0000","last_change":"2023-06-09T04:35:17.435324+0000","last_active":"2023-06-09T04:35:18.423602+0000","last_peered":"2023-06-09T04:35:18.423602+0000","last_clean":"2023-06-09T04:35:18.423602+0000","last_became_active":"2023-06-09T04:35:17.435058+0000","last_became_peered":"2023-06-09T04:35:17.435058+0000","last_unstale":"2023-06-09T04:35:18.423602+0000","last_undegraded":"2023-06-09T04:35:18.423602+0000","last_fullsized":"2023-06-09T04:35:18.423602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:38:44.935250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:19.070935+0000","last_change":"2023-06-09T04:35:17.434438+0000","last_active":"2023-06-09T04:35:19.070935+0000","last_peered":"2023-06-09T04:35:19.070935+0000","last_clean":"2023-06-09T04:35:19.070935+0000","last_became_active":"2023-06-09T04:35:17.432923+0000","last_became_peered":"2023-06-09T04:35:17.432923+0000","last_unstale":"2023-06-09T04:35:19.070935+0000","last_undegraded":"2023-06-09T04:35:19.070935+0000","last_fullsized":"2023-06-09T04:35:19.070935+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:18.424114+0000","last_change":"2023-06-09T04:35:17.431908+0000","last_active":"2023-06-09T04:35:18.424114+0000","last_peered":"2023-06-09T04:35:18.424114+0000","last_clean":"2023-06-09T04:35:18.424114+0000","last_became_active":"2023-06-09T04:35:17.431682+0000","last_became_peered":"2023-06-09T04:35:17.431682+0000","last_unstale":"2023-06-09T04:35:18.424114+0000","last_undegraded":"2023-06-09T04:35:18.424114+0000","last_fullsized":"2023-06-09T04:35:18.424114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:50:54.212660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-06-09T04:35:20.354496+0000","last_change":"2023-06-09T04:35:17.435983+0000","last_active":"2023-06-09T04:35:20.354496+0000","last_peered":"2023-06-09T04:35:20.354496+0000","last_clean":"2023-06-09T04:35:20.354496+0000","last_became_active":"2023-06-09T04:35:17.435723+0000","last_became_peered":"2023-06-09T04:35:17.435723+0000","last_unstale":"2023-06-09T04:35:20.354496+0000","last_undegraded":"2023-06-09T04:35:20.354496+0000","last_fullsized":"2023-06-09T04:35:20.354496+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:17.006709+0000","last_clean_scrub_stamp":"2023-06-09T04:35:17.006709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:58:18.985065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":876544,"data_stored":686418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4449999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":847872,"data_stored":684551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":253952,"data_stored":94270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":258048,"data_stored":94360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":274432,"data_stored":95890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":94922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T04:35:26.499 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-09T04:35:26.500 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-09T04:35:26.500 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-06-09T04:35:26.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-06-09T04:35:26.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.651+0000 7ff048015700 1 Processor -- start 2023-06-09T04:35:26.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.651+0000 7ff048015700 1 -- start start 2023-06-09T04:35:26.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff048015700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0400ce350 0x7ff0400aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff048015700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff040159640 0x7ff0400ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff048015700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04015cda0 con 0x7ff0400713d0 2023-06-09T04:35:26.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff048015700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff040159ab0 con 0x7ff040159640 2023-06-09T04:35:26.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff048015700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400cec60 con 0x7ff0400ce350 2023-06-09T04:35:26.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff040159640 0x7ff0400ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff040159640 0x7ff0400ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38012/0 (socket says 172.21.15.92:38012) 2023-06-09T04:35:26.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 -- 172.21.15.92:0/2823980624 learned_addr learned my addr 172.21.15.92:0/2823980624 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:26.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff045db1700 1 --2- 172.21.15.92:0/2823980624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0400ce350 0x7ff0400aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 -- 172.21.15.92:0/2823980624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0400ce350 msgr2=0x7ff0400aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:26.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 --2- 172.21.15.92:0/2823980624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0400ce350 0x7ff0400aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 -- 172.21.15.92:0/2823980624 >> v1:172.21.15.123:6789/0 conn(0x7ff0400713d0 legacy=0x7ff04015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:26.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 -- 172.21.15.92:0/2823980624 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff040159d40 con 0x7ff040159640 2023-06-09T04:35:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff0455b0700 1 --2- 172.21.15.92:0/2823980624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff040159640 0x7ff0400ce720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0300098f0 tx=0x7ff030009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed8795530aa3482 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.652+0000 7ff044daf700 1 -- 172.21.15.92:0/2823980624 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000b8a0 con 0x7ff040159640 2023-06-09T04:35:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff044daf700 1 -- 172.21.15.92:0/2823980624 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03000ba00 con 0x7ff040159640 2023-06-09T04:35:26.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff044daf700 1 -- 172.21.15.92:0/2823980624 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000bd90 con 0x7ff040159640 2023-06-09T04:35:26.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 -- 172.21.15.92:0/2823980624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff040159640 msgr2=0x7ff0400ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 --2- 172.21.15.92:0/2823980624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff040159640 0x7ff0400ce720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0300098f0 tx=0x7ff030009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:26.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 -- 172.21.15.92:0/2823980624 shutdown_connections 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 --2- 172.21.15.92:0/2823980624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0400ce350 0x7ff0400aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 --2- 172.21.15.92:0/2823980624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff040159640 0x7ff0400ce720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 -- 172.21.15.92:0/2823980624 >> 172.21.15.92:0/2823980624 conn(0x7ff04006d580 msgr2=0x7ff04006d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 -- 172.21.15.92:0/2823980624 shutdown_connections 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.653+0000 7ff048015700 1 -- 172.21.15.92:0/2823980624 wait complete. 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.655+0000 7ff048015700 1 Processor -- start 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.655+0000 7ff048015700 1 -- start start 2023-06-09T04:35:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0400713d0 0x7ff0401f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0400ce350 0x7ff0400b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff040159640 0x7ff040152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400b3750 con 0x7ff0400713d0 2023-06-09T04:35:26.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400b0140 con 0x7ff0400ce350 2023-06-09T04:35:26.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400b09a0 con 0x7ff040159640 2023-06-09T04:35:26.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0400713d0 0x7ff0401f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0400713d0 0x7ff0401f0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38016/0 (socket says 172.21.15.92:38016) 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 -- 172.21.15.92:0/3777080871 learned_addr learned my addr 172.21.15.92:0/3777080871 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff0455b0700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0400ce350 0x7ff0400b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff0465b2700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff040159640 0x7ff040152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 -- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff040159640 msgr2=0x7ff040152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff040159640 0x7ff040152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 -- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0400ce350 msgr2=0x7ff0400b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0400ce350 0x7ff0400b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 -- 172.21.15.92:0/3777080871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030009550 con 0x7ff0400713d0 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff0465b2700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff040159640 0x7ff040152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff0455b0700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0400ce350 0x7ff0400b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff045db1700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0400713d0 0x7ff0401f0110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff03c010700 tx=0x7ff03c010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff0367fc700 1 -- 172.21.15.92:0/3777080871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c011920 con 0x7ff0400713d0 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff0367fc700 1 -- 172.21.15.92:0/3777080871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c011a80 con 0x7ff0400713d0 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff0367fc700 1 -- 172.21.15.92:0/3777080871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c011e30 con 0x7ff0400713d0 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0400b3470 con 0x7ff0400713d0 2023-06-09T04:35:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.656+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0400b0630 con 0x7ff0400713d0 2023-06-09T04:35:26.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.657+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0400b07c0 con 0x7ff0400713d0 2023-06-09T04:35:26.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.658+0000 7ff0367fc700 1 -- 172.21.15.92:0/3777080871 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff03c019430 con 0x7ff0400713d0 2023-06-09T04:35:26.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.658+0000 7ff0367fc700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff02406ace0 0x7ff02406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:26.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.658+0000 7ff0367fc700 1 -- 172.21.15.92:0/3777080871 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff03c08b560 con 0x7ff0400713d0 2023-06-09T04:35:26.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.658+0000 7ff0455b0700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff02406ace0 0x7ff02406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:26.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.658+0000 7ff0455b0700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff02406ace0 0x7ff02406d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff040158990 tx=0x7ff030008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:26.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.662+0000 7ff0367fc700 1 -- 172.21.15.92:0/3777080871 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff03c056620 con 0x7ff0400713d0 2023-06-09T04:35:26.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.892+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff04015a5b0 con 0x7ff0400713d0 2023-06-09T04:35:26.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.892+0000 7ff0367fc700 1 -- 172.21.15.92:0/3777080871 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff03c059c40 con 0x7ff0400713d0 2023-06-09T04:35:26.893 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:26.893 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-09T04:35:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff02406ace0 msgr2=0x7ff02406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff02406ace0 0x7ff02406d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff040158990 tx=0x7ff030008030 comp rx=0 tx=0).stop 2023-06-09T04:35:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0400713d0 msgr2=0x7ff0401f0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0400713d0 0x7ff0401f0110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff03c010700 tx=0x7ff03c010ac0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 shutdown_connections 2023-06-09T04:35:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff02406ace0 0x7ff02406d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff040159640 0x7ff040152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0400713d0 0x7ff0401f0110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 --2- 172.21.15.92:0/3777080871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0400ce350 0x7ff0400b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 >> 172.21.15.92:0/3777080871 conn(0x7ff04006d580 msgr2=0x7ff0400cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 shutdown_connections 2023-06-09T04:35:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:26.894+0000 7ff048015700 1 -- 172.21.15.92:0/3777080871 wait complete. 2023-06-09T04:35:26.906 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-06-09T04:35:26.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.026+0000 7f7e3ac8a700 1 Processor -- start 2023-06-09T04:35:27.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.026+0000 7f7e3ac8a700 1 -- start start 2023-06-09T04:35:27.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.026+0000 7f7e3ac8a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7e3415a8d0 0x7f7e34069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.026+0000 7f7e3ac8a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e3406a2f0 0x7f7e340ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.026+0000 7f7e3ac8a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e340ac400 con 0x7f7e3415a500 2023-06-09T04:35:27.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.026+0000 7f7e3ac8a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3406a790 con 0x7f7e3406a2f0 2023-06-09T04:35:27.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e3ac8a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e340cd7e0 con 0x7f7e3415a8d0 2023-06-09T04:35:27.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e3406a2f0 0x7f7e340ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e3406a2f0 0x7f7e340ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38024/0 (socket says 172.21.15.92:38024) 2023-06-09T04:35:27.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 -- 172.21.15.92:0/1512856554 learned_addr learned my addr 172.21.15.92:0/1512856554 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:27.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e33fff700 1 --2- 172.21.15.92:0/1512856554 >> v2:172.21.15.123:3300/0 conn(0x7f7e3415a8d0 0x7f7e34069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 -- 172.21.15.92:0/1512856554 >> v2:172.21.15.123:3300/0 conn(0x7f7e3415a8d0 msgr2=0x7f7e34069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:27.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 --2- 172.21.15.92:0/1512856554 >> v2:172.21.15.123:3300/0 conn(0x7f7e3415a8d0 0x7f7e34069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 -- 172.21.15.92:0/1512856554 >> v1:172.21.15.123:6789/0 conn(0x7f7e3415a500 legacy=0x7f7e340cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 -- 172.21.15.92:0/1512856554 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e340cdda0 con 0x7f7e3406a2f0 2023-06-09T04:35:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e39227700 1 --2- 172.21.15.92:0/1512856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3406a2f0 0x7f7e340ce230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e3406c220 tx=0x7f7e2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12246687df33ab7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e337fe700 1 -- 172.21.15.92:0/1512856554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00e070 con 0x7f7e3406a2f0 2023-06-09T04:35:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e337fe700 1 -- 172.21.15.92:0/1512856554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c003a20 con 0x7f7e3406a2f0 2023-06-09T04:35:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.027+0000 7f7e337fe700 1 -- 172.21.15.92:0/1512856554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c003dd0 con 0x7f7e3406a2f0 2023-06-09T04:35:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/1512856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3406a2f0 msgr2=0x7f7e340ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/1512856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3406a2f0 0x7f7e340ce230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e3406c220 tx=0x7f7e2c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/1512856554 shutdown_connections 2023-06-09T04:35:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/1512856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3406a2f0 0x7f7e340ce230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/1512856554 >> v2:172.21.15.123:3300/0 conn(0x7f7e3415a8d0 0x7f7e34069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/1512856554 >> 172.21.15.92:0/1512856554 conn(0x7f7e34098c10 msgr2=0x7f7e340cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/1512856554 shutdown_connections 2023-06-09T04:35:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.028+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/1512856554 wait complete. 2023-06-09T04:35:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.029+0000 7f7e3ac8a700 1 Processor -- start 2023-06-09T04:35:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.029+0000 7f7e3ac8a700 1 -- start start 2023-06-09T04:35:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e3ac8a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e3406a2f0 0x7f7e340c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e3ac8a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3415a500 0x7f7e341f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e3ac8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e3415a8d0 0x7f7e340b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e3ac8a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340b1940 con 0x7f7e3415a500 2023-06-09T04:35:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e3ac8a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e341f1c30 con 0x7f7e3415a8d0 2023-06-09T04:35:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e3ac8a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e340c83a0 con 0x7f7e3406a2f0 2023-06-09T04:35:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3415a500 0x7f7e341f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3415a500 0x7f7e341f16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38028/0 (socket says 172.21.15.92:38028) 2023-06-09T04:35:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 -- 172.21.15.92:0/781650693 learned_addr learned my addr 172.21.15.92:0/781650693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e39227700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e3415a8d0 0x7f7e340b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e38a26700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e3406a2f0 0x7f7e340c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 -- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e3406a2f0 msgr2=0x7f7e340c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e3406a2f0 0x7f7e340c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 -- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e3415a8d0 msgr2=0x7f7e340b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e3415a8d0 0x7f7e340b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 -- 172.21.15.92:0/781650693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c009b30 con 0x7f7e3415a500 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e38a26700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e3406a2f0 0x7f7e340c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e33fff700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3415a500 0x7f7e341f16f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000d800 tx=0x7f7e2000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.030+0000 7f7e317fa700 1 -- 172.21.15.92:0/781650693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e200125f0 con 0x7f7e3415a500 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.031+0000 7f7e317fa700 1 -- 172.21.15.92:0/781650693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e20012750 con 0x7f7e3415a500 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.031+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e340c8680 con 0x7f7e3415a500 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.031+0000 7f7e317fa700 1 -- 172.21.15.92:0/781650693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20012ae0 con 0x7f7e3415a500 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.031+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e340c8c40 con 0x7f7e3415a500 2023-06-09T04:35:27.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.032+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e34066510 con 0x7f7e3415a500 2023-06-09T04:35:27.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.032+0000 7f7e317fa700 1 -- 172.21.15.92:0/781650693 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e20012c40 con 0x7f7e3415a500 2023-06-09T04:35:27.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.032+0000 7f7e317fa700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7e1806ace0 0x7f7e1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.033+0000 7f7e317fa700 1 -- 172.21.15.92:0/781650693 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e200895e0 con 0x7f7e3415a500 2023-06-09T04:35:27.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.033+0000 7f7e38a26700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7e1806ace0 0x7f7e1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.033+0000 7f7e38a26700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7e1806ace0 0x7f7e1806d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e28007fd0 tx=0x7f7e28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.037+0000 7f7e317fa700 1 -- 172.21.15.92:0/781650693 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e20054690 con 0x7f7e3415a500 2023-06-09T04:35:27.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.227+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e340740e0 con 0x7f7e3415a500 2023-06-09T04:35:27.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.228+0000 7f7e317fa700 1 -- 172.21.15.92:0/781650693 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f7e20057cb0 con 0x7f7e3415a500 2023-06-09T04:35:27.229 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:27.229 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:27.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7e1806ace0 msgr2=0x7f7e1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7e1806ace0 0x7f7e1806d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e28007fd0 tx=0x7f7e28009040 comp rx=0 tx=0).stop 2023-06-09T04:35:27.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3415a500 msgr2=0x7f7e341f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3415a500 0x7f7e341f16f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000d800 tx=0x7f7e2000db10 comp rx=0 tx=0).stop 2023-06-09T04:35:27.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 shutdown_connections 2023-06-09T04:35:27.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f7e1806ace0 0x7f7e1806d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e3415a8d0 0x7f7e340b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7e3406a2f0 0x7f7e340c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 --2- 172.21.15.92:0/781650693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e3415a500 0x7f7e341f16f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 >> 172.21.15.92:0/781650693 conn(0x7f7e34098c10 msgr2=0x7f7e340cafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 shutdown_connections 2023-06-09T04:35:27.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.230+0000 7f7e3ac8a700 1 -- 172.21.15.92:0/781650693 wait complete. 2023-06-09T04:35:27.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:27.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:27.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.366+0000 7f575d660700 1 Processor -- start 2023-06-09T04:35:27.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.366+0000 7f575d660700 1 -- start start 2023-06-09T04:35:27.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f575d660700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f57580b1df0 0x7f57580b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f575d660700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57580cd940 0x7f57580d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f575d660700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5758161090 con 0x7f57580b4ba0 2023-06-09T04:35:27.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f575d660700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57580b5120 con 0x7f57580cd940 2023-06-09T04:35:27.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f575d660700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57581602d0 con 0x7f57580b1df0 2023-06-09T04:35:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57580cd940 0x7f57580d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57580cd940 0x7f57580d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38038/0 (socket says 172.21.15.92:38038) 2023-06-09T04:35:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 -- 172.21.15.92:0/2160617110 learned_addr learned my addr 172.21.15.92:0/2160617110 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57567fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f57580b4ba0 0x7f57580cd230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36368/0 (socket says 172.21.15.92:36368) 2023-06-09T04:35:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f5756ffd700 1 --2- 172.21.15.92:0/2160617110 >> v2:172.21.15.123:3300/0 conn(0x7f57580b1df0 0x7f57580b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 -- 172.21.15.92:0/2160617110 >> v2:172.21.15.123:3300/0 conn(0x7f57580b1df0 msgr2=0x7f57580b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 --2- 172.21.15.92:0/2160617110 >> v2:172.21.15.123:3300/0 conn(0x7f57580b1df0 0x7f57580b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 -- 172.21.15.92:0/2160617110 >> v1:172.21.15.92:6789/0 conn(0x7f57580b4ba0 legacy=0x7f57580cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 -- 172.21.15.92:0/2160617110 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57581605b0 con 0x7f57580cd940 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f5755ffb700 1 -- 172.21.15.92:0/2160617110 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318128343 0 0) 0x7f5758161090 con 0x7f57580b4ba0 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f5756ffd700 1 --2- 172.21.15.92:0/2160617110 >> v2:172.21.15.123:3300/0 conn(0x7f57580b1df0 0x7f57580b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.367+0000 7f57577fe700 1 --2- 172.21.15.92:0/2160617110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580cd940 0x7f57580d0a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f575806a140 tx=0x7f575000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55ee1273aa7b746 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f5755ffb700 1 -- 172.21.15.92:0/2160617110 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000e070 con 0x7f57580cd940 2023-06-09T04:35:27.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f5755ffb700 1 -- 172.21.15.92:0/2160617110 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575000b9c0 con 0x7f57580cd940 2023-06-09T04:35:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f5755ffb700 1 -- 172.21.15.92:0/2160617110 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000bd70 con 0x7f57580cd940 2023-06-09T04:35:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 -- 172.21.15.92:0/2160617110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580cd940 msgr2=0x7f57580d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 --2- 172.21.15.92:0/2160617110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580cd940 0x7f57580d0a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f575806a140 tx=0x7f575000b330 comp rx=0 tx=0).stop 2023-06-09T04:35:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 -- 172.21.15.92:0/2160617110 shutdown_connections 2023-06-09T04:35:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 --2- 172.21.15.92:0/2160617110 >> v2:172.21.15.123:3300/0 conn(0x7f57580b1df0 0x7f57580b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 --2- 172.21.15.92:0/2160617110 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580cd940 0x7f57580d0a50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 -- 172.21.15.92:0/2160617110 >> 172.21.15.92:0/2160617110 conn(0x7f57580666b0 msgr2=0x7f57580cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 -- 172.21.15.92:0/2160617110 shutdown_connections 2023-06-09T04:35:27.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.368+0000 7f575d660700 1 -- 172.21.15.92:0/2160617110 wait complete. 2023-06-09T04:35:27.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.369+0000 7f575d660700 1 Processor -- start 2023-06-09T04:35:27.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.369+0000 7f575d660700 1 -- start start 2023-06-09T04:35:27.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f575d660700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57580b1df0 0x7f5758152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f575d660700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580b4ba0 0x7f57580b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f575d660700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57580cd940 0x7f57580b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f575d660700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5758143860 con 0x7f57580b4ba0 2023-06-09T04:35:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f575d660700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5758142c70 con 0x7f57580cd940 2023-06-09T04:35:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f575d660700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5758142ec0 con 0x7f57580b1df0 2023-06-09T04:35:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f57567fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580b4ba0 0x7f57580b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f57567fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580b4ba0 0x7f57580b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38042/0 (socket says 172.21.15.92:38042) 2023-06-09T04:35:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f57567fc700 1 -- 172.21.15.92:0/2068406939 learned_addr learned my addr 172.21.15.92:0/2068406939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f5756ffd700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57580b1df0 0x7f5758152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.370+0000 7f57577fe700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57580cd940 0x7f57580b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f57567fc700 1 -- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57580b1df0 msgr2=0x7f5758152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f57567fc700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57580b1df0 0x7f5758152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f57567fc700 1 -- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57580cd940 msgr2=0x7f57580b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f57567fc700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57580cd940 0x7f57580b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f57567fc700 1 -- 172.21.15.92:0/2068406939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5750009b30 con 0x7f57580b4ba0 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f57567fc700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580b4ba0 0x7f57580b3470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57480011e0 tx=0x7f574800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f573f7fe700 1 -- 172.21.15.92:0/2068406939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748004c80 con 0x7f57580b4ba0 2023-06-09T04:35:27.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f573f7fe700 1 -- 172.21.15.92:0/2068406939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5748004e00 con 0x7f57580b4ba0 2023-06-09T04:35:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57581431a0 con 0x7f57580b4ba0 2023-06-09T04:35:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f573f7fe700 1 -- 172.21.15.92:0/2068406939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748007310 con 0x7f57580b4ba0 2023-06-09T04:35:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.371+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5758153450 con 0x7f57580b4ba0 2023-06-09T04:35:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.372+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57581439a0 con 0x7f57580b4ba0 2023-06-09T04:35:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.372+0000 7f573f7fe700 1 -- 172.21.15.92:0/2068406939 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5748007470 con 0x7f57580b4ba0 2023-06-09T04:35:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.373+0000 7f573f7fe700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f574406ace0 0x7f574406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.373+0000 7f5756ffd700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f574406ace0 0x7f574406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.373+0000 7f573f7fe700 1 -- 172.21.15.92:0/2068406939 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5748089040 con 0x7f57580b4ba0 2023-06-09T04:35:27.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.373+0000 7f5756ffd700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f574406ace0 0x7f574406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f574c006f10 tx=0x7f574c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.377+0000 7f573f7fe700 1 -- 172.21.15.92:0/2068406939 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57581439a0 con 0x7f57580b4ba0 2023-06-09T04:35:27.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.560+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57581439a0 con 0x7f57580b4ba0 2023-06-09T04:35:27.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.561+0000 7f573f7fe700 1 -- 172.21.15.92:0/2068406939 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57581439a0 con 0x7f57580b4ba0 2023-06-09T04:35:27.561 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:27.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f574406ace0 msgr2=0x7f574406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f574406ace0 0x7f574406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f574c006f10 tx=0x7f574c008040 comp rx=0 tx=0).stop 2023-06-09T04:35:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580b4ba0 msgr2=0x7f57580b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580b4ba0 0x7f57580b3470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57480011e0 tx=0x7f574800ca30 comp rx=0 tx=0).stop 2023-06-09T04:35:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 shutdown_connections 2023-06-09T04:35:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f574406ace0 0x7f574406d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57580cd940 0x7f57580b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57580b1df0 0x7f5758152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 --2- 172.21.15.92:0/2068406939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57580b4ba0 0x7f57580b3470 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 >> 172.21.15.92:0/2068406939 conn(0x7f57580666b0 msgr2=0x7f575815f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 shutdown_connections 2023-06-09T04:35:27.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.563+0000 7f575d660700 1 -- 172.21.15.92:0/2068406939 wait complete. 2023-06-09T04:35:27.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:27.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.697+0000 7fe31fe21700 1 Processor -- start 2023-06-09T04:35:27.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.697+0000 7fe31fe21700 1 -- start start 2023-06-09T04:35:27.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31fe21700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe318070a30 0x7fe31806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31fe21700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe3181600b0 0x7fe31815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31fe21700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3180cd550 con 0x7fe3180b16e0 2023-06-09T04:35:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31fe21700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31806b290 con 0x7fe318070a30 2023-06-09T04:35:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31fe21700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3180ccf90 con 0x7fe3181600b0 2023-06-09T04:35:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31dbbd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe318070a30 0x7fe31806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31dbbd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe318070a30 0x7fe31806d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38048/0 (socket says 172.21.15.92:38048) 2023-06-09T04:35:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31dbbd700 1 -- 172.21.15.92:0/3715766366 learned_addr learned my addr 172.21.15.92:0/3715766366 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31d3bc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe3180b16e0 0x7fe318070e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36380/0 (socket says 172.21.15.92:36380) 2023-06-09T04:35:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31e3be700 1 --2- 172.21.15.92:0/3715766366 >> v2:172.21.15.123:3300/0 conn(0x7fe3181600b0 0x7fe31815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31dbbd700 1 -- 172.21.15.92:0/3715766366 >> v2:172.21.15.123:3300/0 conn(0x7fe3181600b0 msgr2=0x7fe31815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31dbbd700 1 --2- 172.21.15.92:0/3715766366 >> v2:172.21.15.123:3300/0 conn(0x7fe3181600b0 0x7fe31815ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31dbbd700 1 -- 172.21.15.92:0/3715766366 >> v1:172.21.15.92:6789/0 conn(0x7fe3180b16e0 legacy=0x7fe318070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31dbbd700 1 -- 172.21.15.92:0/3715766366 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180cd270 con 0x7fe318070a30 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.698+0000 7fe31cbbb700 1 -- 172.21.15.92:0/3715766366 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894091651 0 0) 0x7fe3180cd550 con 0x7fe3180b16e0 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31dbbd700 1 --2- 172.21.15.92:0/3715766366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe318070a30 0x7fe31806d5c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe3180cba20 tx=0x7fe308009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e96bc835a0b1fb9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31cbbb700 1 -- 172.21.15.92:0/3715766366 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800b870 con 0x7fe318070a30 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31cbbb700 1 -- 172.21.15.92:0/3715766366 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30800b9d0 con 0x7fe318070a30 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31cbbb700 1 -- 172.21.15.92:0/3715766366 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800bd60 con 0x7fe318070a30 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 -- 172.21.15.92:0/3715766366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe318070a30 msgr2=0x7fe31806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 --2- 172.21.15.92:0/3715766366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe318070a30 0x7fe31806d5c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe3180cba20 tx=0x7fe308009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 -- 172.21.15.92:0/3715766366 shutdown_connections 2023-06-09T04:35:27.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 --2- 172.21.15.92:0/3715766366 >> v2:172.21.15.123:3300/0 conn(0x7fe3181600b0 0x7fe31815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 --2- 172.21.15.92:0/3715766366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe318070a30 0x7fe31806d5c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 -- 172.21.15.92:0/3715766366 >> 172.21.15.92:0/3715766366 conn(0x7fe3180cfc40 msgr2=0x7fe3180b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:27.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 -- 172.21.15.92:0/3715766366 shutdown_connections 2023-06-09T04:35:27.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.699+0000 7fe31fe21700 1 -- 172.21.15.92:0/3715766366 wait complete. 2023-06-09T04:35:27.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.700+0000 7fe31fe21700 1 Processor -- start 2023-06-09T04:35:27.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.700+0000 7fe31fe21700 1 -- start start 2023-06-09T04:35:27.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31fe21700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318070a30 0x7fe3181563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31fe21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3180b16e0 0x7fe318159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31fe21700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3181600b0 0x7fe31815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31fe21700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181e5f10 con 0x7fe3180b16e0 2023-06-09T04:35:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31fe21700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181f2540 con 0x7fe318070a30 2023-06-09T04:35:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31fe21700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181f2820 con 0x7fe3181600b0 2023-06-09T04:35:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31d3bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3180b16e0 0x7fe318159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31d3bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3180b16e0 0x7fe318159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38054/0 (socket says 172.21.15.92:38054) 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31d3bc700 1 -- 172.21.15.92:0/4195119202 learned_addr learned my addr 172.21.15.92:0/4195119202 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31dbbd700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318070a30 0x7fe3181563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.701+0000 7fe31e3be700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3181600b0 0x7fe31815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31d3bc700 1 -- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3181600b0 msgr2=0x7fe31815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31d3bc700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3181600b0 0x7fe31815b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31d3bc700 1 -- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318070a30 msgr2=0x7fe3181563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31d3bc700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318070a30 0x7fe3181563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31d3bc700 1 -- 172.21.15.92:0/4195119202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009550 con 0x7fe3180b16e0 2023-06-09T04:35:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31d3bc700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3180b16e0 0x7fe318159c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c550 tx=0x7fe31400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe30e7fc700 1 -- 172.21.15.92:0/4195119202 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314004c80 con 0x7fe3180b16e0 2023-06-09T04:35:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe30e7fc700 1 -- 172.21.15.92:0/4195119202 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe314004e00 con 0x7fe3180b16e0 2023-06-09T04:35:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3181f2b00 con 0x7fe3180b16e0 2023-06-09T04:35:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe30e7fc700 1 -- 172.21.15.92:0/4195119202 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314007310 con 0x7fe3180b16e0 2023-06-09T04:35:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.702+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31815a1c0 con 0x7fe3180b16e0 2023-06-09T04:35:27.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.703+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3181e60a0 con 0x7fe3180b16e0 2023-06-09T04:35:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.703+0000 7fe30e7fc700 1 -- 172.21.15.92:0/4195119202 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe314007470 con 0x7fe3180b16e0 2023-06-09T04:35:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.704+0000 7fe30e7fc700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe2fc06ace0 0x7fe2fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.704+0000 7fe30e7fc700 1 -- 172.21.15.92:0/4195119202 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe314089190 con 0x7fe3180b16e0 2023-06-09T04:35:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.704+0000 7fe31dbbd700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe2fc06ace0 0x7fe2fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.704+0000 7fe31dbbd700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe2fc06ace0 0x7fe2fc06d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe308014040 tx=0x7fe308008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.708+0000 7fe30e7fc700 1 -- 172.21.15.92:0/4195119202 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3181e60a0 con 0x7fe3180b16e0 2023-06-09T04:35:27.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.895+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3181e60a0 con 0x7fe3180b16e0 2023-06-09T04:35:27.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.895+0000 7fe30e7fc700 1 -- 172.21.15.92:0/4195119202 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe3181e60a0 con 0x7fe3180b16e0 2023-06-09T04:35:27.896 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:27.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.897+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe2fc06ace0 msgr2=0x7fe2fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.897+0000 7fe31fe21700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe2fc06ace0 0x7fe2fc06d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe308014040 tx=0x7fe308008030 comp rx=0 tx=0).stop 2023-06-09T04:35:27.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.897+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3180b16e0 msgr2=0x7fe318159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:27.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.897+0000 7fe31fe21700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3180b16e0 0x7fe318159c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c550 tx=0x7fe31400c910 comp rx=0 tx=0).stop 2023-06-09T04:35:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.897+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 shutdown_connections 2023-06-09T04:35:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.897+0000 7fe31fe21700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe2fc06ace0 0x7fe2fc06d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.897+0000 7fe31fe21700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3181600b0 0x7fe31815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.898+0000 7fe31fe21700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe318070a30 0x7fe3181563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.898+0000 7fe31fe21700 1 --2- 172.21.15.92:0/4195119202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3180b16e0 0x7fe318159c80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.898+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 >> 172.21.15.92:0/4195119202 conn(0x7fe3180cfc40 msgr2=0x7fe31815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:27.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.898+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 shutdown_connections 2023-06-09T04:35:27.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:27.898+0000 7fe31fe21700 1 -- 172.21.15.92:0/4195119202 wait complete. 2023-06-09T04:35:27.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:28.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.033+0000 7f52ca5ae700 1 Processor -- start 2023-06-09T04:35:28.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.033+0000 7f52ca5ae700 1 -- start start 2023-06-09T04:35:28.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52ca5ae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aee60 0x7f52c40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52ca5ae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52c406a4c0 0x7f52c406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52ca5ae700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c4071dc0 con 0x7f52c40aea90 2023-06-09T04:35:28.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52ca5ae700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c415edf0 con 0x7f52c406a4c0 2023-06-09T04:35:28.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52ca5ae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40cf030 con 0x7f52c40aee60 2023-06-09T04:35:28.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52c8b4b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52c406a4c0 0x7f52c406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52c8b4b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52c406a4c0 0x7f52c406ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38064/0 (socket says 172.21.15.92:38064) 2023-06-09T04:35:28.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52c8b4b700 1 -- 172.21.15.92:0/564885937 learned_addr learned my addr 172.21.15.92:0/564885937 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:28.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52c37fe700 1 --2- 172.21.15.92:0/564885937 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aee60 0x7f52c40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52c8b4b700 1 -- 172.21.15.92:0/564885937 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aee60 msgr2=0x7f52c40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52c8b4b700 1 --2- 172.21.15.92:0/564885937 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aee60 0x7f52c40cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.034+0000 7f52c8b4b700 1 -- 172.21.15.92:0/564885937 >> v1:172.21.15.123:6789/0 conn(0x7f52c40aea90 legacy=0x7f52c4069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52c8b4b700 1 -- 172.21.15.92:0/564885937 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40cf2d0 con 0x7f52c406a4c0 2023-06-09T04:35:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52c8b4b700 1 --2- 172.21.15.92:0/564885937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c406a4c0 0x7f52c406ee20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009e50 tx=0x7f52bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b966d18f34fb22d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52c2ffd700 1 -- 172.21.15.92:0/564885937 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00e070 con 0x7f52c406a4c0 2023-06-09T04:35:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52c2ffd700 1 -- 172.21.15.92:0/564885937 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc00b910 con 0x7f52c406a4c0 2023-06-09T04:35:28.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52c2ffd700 1 -- 172.21.15.92:0/564885937 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00bcc0 con 0x7f52c406a4c0 2023-06-09T04:35:28.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 -- 172.21.15.92:0/564885937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c406a4c0 msgr2=0x7f52c406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/564885937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c406a4c0 0x7f52c406ee20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009e50 tx=0x7f52bc00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:28.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 -- 172.21.15.92:0/564885937 shutdown_connections 2023-06-09T04:35:28.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/564885937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c406a4c0 0x7f52c406ee20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/564885937 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aee60 0x7f52c40cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 -- 172.21.15.92:0/564885937 >> 172.21.15.92:0/564885937 conn(0x7f52c40cc380 msgr2=0x7f52c40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:28.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 -- 172.21.15.92:0/564885937 shutdown_connections 2023-06-09T04:35:28.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.035+0000 7f52ca5ae700 1 -- 172.21.15.92:0/564885937 wait complete. 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 Processor -- start 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 -- start start 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c406a4c0 0x7f52c415a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aea90 0x7f52c4147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c40c46c0 0x7f52c40c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40c5960 con 0x7f52c40c46c0 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4148340 con 0x7f52c406a4c0 2023-06-09T04:35:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.037+0000 7f52ca5ae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40c62c0 con 0x7f52c40aea90 2023-06-09T04:35:28.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c40c46c0 0x7f52c40c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c40c46c0 0x7f52c40c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38070/0 (socket says 172.21.15.92:38070) 2023-06-09T04:35:28.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 -- 172.21.15.92:0/1925640669 learned_addr learned my addr 172.21.15.92:0/1925640669 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:28.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c37fe700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aea90 0x7f52c4147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c3fff700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c406a4c0 0x7f52c415a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 -- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aea90 msgr2=0x7f52c4147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aea90 0x7f52c4147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 -- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c406a4c0 msgr2=0x7f52c415a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c406a4c0 0x7f52c415a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 -- 172.21.15.92:0/1925640669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009b30 con 0x7f52c40c46c0 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c8b4b700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c40c46c0 0x7f52c40c5d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52bc012000 tx=0x7f52bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c0ff9700 1 -- 172.21.15.92:0/1925640669 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc004800 con 0x7f52c40c46c0 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c0ff9700 1 -- 172.21.15.92:0/1925640669 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc004960 con 0x7f52c40c46c0 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c40c5050 con 0x7f52c40c46c0 2023-06-09T04:35:28.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52c0ff9700 1 -- 172.21.15.92:0/1925640669 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc01b670 con 0x7f52c40c46c0 2023-06-09T04:35:28.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.038+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c40c52a0 con 0x7f52c40c46c0 2023-06-09T04:35:28.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.039+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c40c5aa0 con 0x7f52c40c46c0 2023-06-09T04:35:28.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.039+0000 7f52c0ff9700 1 -- 172.21.15.92:0/1925640669 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52bc01b7d0 con 0x7f52c40c46c0 2023-06-09T04:35:28.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.040+0000 7f52c0ff9700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f52b006ace0 0x7f52b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.040+0000 7f52c0ff9700 1 -- 172.21.15.92:0/1925640669 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52bc08ab50 con 0x7f52c40c46c0 2023-06-09T04:35:28.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.040+0000 7f52c3fff700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f52b006ace0 0x7f52b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.040+0000 7f52c3fff700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f52b006ace0 0x7f52b006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f52b8007fd0 tx=0x7f52b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.044+0000 7f52c0ff9700 1 -- 172.21.15.92:0/1925640669 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f52bc055c90 con 0x7f52c40c46c0 2023-06-09T04:35:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.232+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52c404e8e0 con 0x7f52c40c46c0 2023-06-09T04:35:28.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.233+0000 7f52c0ff9700 1 -- 172.21.15.92:0/1925640669 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52bc0592b0 con 0x7f52c40c46c0 2023-06-09T04:35:28.234 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:28.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f52b006ace0 msgr2=0x7f52b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f52b006ace0 0x7f52b006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f52b8007fd0 tx=0x7f52b8009040 comp rx=0 tx=0).stop 2023-06-09T04:35:28.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c40c46c0 msgr2=0x7f52c40c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c40c46c0 0x7f52c40c5d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52bc012000 tx=0x7f52bc004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 shutdown_connections 2023-06-09T04:35:28.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f52b006ace0 0x7f52b006d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c40c46c0 0x7f52c40c5d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c406a4c0 0x7f52c415a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 --2- 172.21.15.92:0/1925640669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52c40aea90 0x7f52c4147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 >> 172.21.15.92:0/1925640669 conn(0x7f52c40cc380 msgr2=0x7f52c40b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:28.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 shutdown_connections 2023-06-09T04:35:28.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.235+0000 7f52ca5ae700 1 -- 172.21.15.92:0/1925640669 wait complete. 2023-06-09T04:35:28.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:28.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.369+0000 7f567eb62700 1 Processor -- start 2023-06-09T04:35:28.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.369+0000 7f567eb62700 1 -- start start 2023-06-09T04:35:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567eb62700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f567815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567eb62700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56780cce60 0x7f567806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567eb62700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567815d040 con 0x7f56780cf440 2023-06-09T04:35:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567eb62700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5678160ab0 con 0x7f56780cce60 2023-06-09T04:35:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567eb62700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567815ca10 con 0x7f5678069db0 2023-06-09T04:35:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567d0ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56780cce60 0x7f567806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f5677fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f56780cf440 0x7f567806a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36406/0 (socket says 172.21.15.92:36406) 2023-06-09T04:35:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f5677fff700 1 -- 172.21.15.92:0/891470856 learned_addr learned my addr 172.21.15.92:0/891470856 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567c8fe700 1 --2- 172.21.15.92:0/891470856 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f567815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567d0ff700 1 -- 172.21.15.92:0/891470856 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 msgr2=0x7f567815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:28.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567d0ff700 1 --2- 172.21.15.92:0/891470856 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f567815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567d0ff700 1 -- 172.21.15.92:0/891470856 >> v1:172.21.15.92:6789/0 conn(0x7f56780cf440 legacy=0x7f567806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f56777fe700 1 -- 172.21.15.92:0/891470856 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422873290 0 0) 0x7f567815d040 con 0x7f56780cf440 2023-06-09T04:35:28.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.370+0000 7f567d0ff700 1 -- 172.21.15.92:0/891470856 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678160d50 con 0x7f56780cce60 2023-06-09T04:35:28.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567d0ff700 1 --2- 172.21.15.92:0/891470856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f567806ee20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5670009d90 tx=0x7f567000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c294921245b4c748 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f56777fe700 1 -- 172.21.15.92:0/891470856 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000e070 con 0x7f56780cce60 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f56777fe700 1 -- 172.21.15.92:0/891470856 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567000bb20 con 0x7f56780cce60 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f56777fe700 1 -- 172.21.15.92:0/891470856 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670012430 con 0x7f56780cce60 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 -- 172.21.15.92:0/891470856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 msgr2=0x7f567806ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 --2- 172.21.15.92:0/891470856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f567806ee20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5670009d90 tx=0x7f567000b330 comp rx=0 tx=0).stop 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 -- 172.21.15.92:0/891470856 shutdown_connections 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 --2- 172.21.15.92:0/891470856 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f567815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 --2- 172.21.15.92:0/891470856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f567806ee20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 -- 172.21.15.92:0/891470856 >> 172.21.15.92:0/891470856 conn(0x7f56780cc380 msgr2=0x7f56780cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:28.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 -- 172.21.15.92:0/891470856 shutdown_connections 2023-06-09T04:35:28.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.371+0000 7f567eb62700 1 -- 172.21.15.92:0/891470856 wait complete. 2023-06-09T04:35:28.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.372+0000 7f567eb62700 1 Processor -- start 2023-06-09T04:35:28.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.372+0000 7f567eb62700 1 -- start start 2023-06-09T04:35:28.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f567eb62700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f5678155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f567eb62700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f56780ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f567eb62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56780cf440 0x7f56780b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f567eb62700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56780b0a20 con 0x7f56780cce60 2023-06-09T04:35:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f567eb62700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56780b0d00 con 0x7f56780cf440 2023-06-09T04:35:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f567eb62700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56780b0fe0 con 0x7f5678069db0 2023-06-09T04:35:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f5677fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f56780ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f5677fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f56780ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38080/0 (socket says 172.21.15.92:38080) 2023-06-09T04:35:28.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.373+0000 7f5677fff700 1 -- 172.21.15.92:0/1759452356 learned_addr learned my addr 172.21.15.92:0/1759452356 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:28.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f567c8fe700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f5678155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f567d0ff700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56780cf440 0x7f56780b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f5677fff700 1 -- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 msgr2=0x7f5678155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f5677fff700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f5678155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f5677fff700 1 -- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56780cf440 msgr2=0x7f56780b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f5677fff700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56780cf440 0x7f56780b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f5677fff700 1 -- 172.21.15.92:0/1759452356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670009a70 con 0x7f56780cce60 2023-06-09T04:35:28.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f567c8fe700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f5678155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:28.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f5677fff700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f56780ad020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f566400c550 tx=0x7f566400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f56757fa700 1 -- 172.21.15.92:0/1759452356 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664004c80 con 0x7f56780cce60 2023-06-09T04:35:28.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f56757fa700 1 -- 172.21.15.92:0/1759452356 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5664004e00 con 0x7f56780cce60 2023-06-09T04:35:28.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f56757fa700 1 -- 172.21.15.92:0/1759452356 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664007310 con 0x7f56780cce60 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56780b12c0 con 0x7f56780cce60 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.374+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5678155be0 con 0x7f56780cce60 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.375+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56780b17e0 con 0x7f56780cce60 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.375+0000 7f56757fa700 1 -- 172.21.15.92:0/1759452356 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5664007470 con 0x7f56780cce60 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.376+0000 7f56757fa700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f565c06ace0 0x7f565c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.376+0000 7f56757fa700 1 -- 172.21.15.92:0/1759452356 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56640891c0 con 0x7f56780cce60 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.376+0000 7f567c8fe700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f565c06ace0 0x7f565c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.376+0000 7f567c8fe700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f565c06ace0 0x7f565c06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f566c009c30 tx=0x7f566c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.380+0000 7f56757fa700 1 -- 172.21.15.92:0/1759452356 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f56780b17e0 con 0x7f56780cce60 2023-06-09T04:35:28.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.566+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56780b17e0 con 0x7f56780cce60 2023-06-09T04:35:28.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.567+0000 7f56757fa700 1 -- 172.21.15.92:0/1759452356 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56780b17e0 con 0x7f56780cce60 2023-06-09T04:35:28.567 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:28.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.568+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f565c06ace0 msgr2=0x7f565c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.568+0000 7f567eb62700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f565c06ace0 0x7f565c06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f566c009c30 tx=0x7f566c008040 comp rx=0 tx=0).stop 2023-06-09T04:35:28.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.568+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 msgr2=0x7f56780ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.568+0000 7f567eb62700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f56780ad020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f566400c550 tx=0x7f566400c910 comp rx=0 tx=0).stop 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 shutdown_connections 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f565c06ace0 0x7f565c06d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56780cf440 0x7f56780b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5678069db0 0x7f5678155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 --2- 172.21.15.92:0/1759452356 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56780cce60 0x7f56780ad020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 >> 172.21.15.92:0/1759452356 conn(0x7f56780cc380 msgr2=0x7f567815fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 shutdown_connections 2023-06-09T04:35:28.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.569+0000 7f567eb62700 1 -- 172.21.15.92:0/1759452356 wait complete. 2023-06-09T04:35:28.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:35:28.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.699+0000 7f4be83a0700 1 Processor -- start 2023-06-09T04:35:28.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.699+0000 7f4be83a0700 1 -- start start 2023-06-09T04:35:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.700+0000 7f4be83a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4be00b1360 0x7f4be00b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.700+0000 7f4be83a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ae630 0x7f4be00aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.700+0000 7f4be83a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00adde0 con 0x7f4be00ac400 2023-06-09T04:35:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.700+0000 7f4be83a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be00ae270 con 0x7f4be00b1360 2023-06-09T04:35:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.700+0000 7f4be83a0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00af0f0 con 0x7f4be00ae630 2023-06-09T04:35:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.700+0000 7f4be613c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4be00b1360 0x7f4be00b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be693d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4be00ac400 0x7f4be00ac880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36424/0 (socket says 172.21.15.92:36424) 2023-06-09T04:35:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be613c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4be00b1360 0x7f4be00b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38088/0 (socket says 172.21.15.92:38088) 2023-06-09T04:35:28.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be693d700 1 -- 172.21.15.92:0/3566581802 learned_addr learned my addr 172.21.15.92:0/3566581802 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be593b700 1 --2- 172.21.15.92:0/3566581802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ae630 0x7f4be00aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be613c700 1 -- 172.21.15.92:0/3566581802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ae630 msgr2=0x7f4be00aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be613c700 1 --2- 172.21.15.92:0/3566581802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ae630 0x7f4be00aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be613c700 1 -- 172.21.15.92:0/3566581802 >> v1:172.21.15.92:6789/0 conn(0x7f4be00ac400 legacy=0x7f4be00ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be613c700 1 -- 172.21.15.92:0/3566581802 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00b1c70 con 0x7f4be00b1360 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be513a700 1 -- 172.21.15.92:0/3566581802 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448764451 0 0) 0x7f4be00adde0 con 0x7f4be00ac400 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be613c700 1 --2- 172.21.15.92:0/3566581802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00b1730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009820 tx=0x7f4bd4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a714626dbdd67 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be513a700 1 -- 172.21.15.92:0/3566581802 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4005e30 con 0x7f4be00b1360 2023-06-09T04:35:28.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be513a700 1 -- 172.21.15.92:0/3566581802 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd4006030 con 0x7f4be00b1360 2023-06-09T04:35:28.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.701+0000 7f4be513a700 1 -- 172.21.15.92:0/3566581802 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40063e0 con 0x7f4be00b1360 2023-06-09T04:35:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 -- 172.21.15.92:0/3566581802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 msgr2=0x7f4be00b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3566581802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00b1730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009820 tx=0x7f4bd4009b30 comp rx=0 tx=0).stop 2023-06-09T04:35:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 -- 172.21.15.92:0/3566581802 shutdown_connections 2023-06-09T04:35:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3566581802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00b1730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3566581802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ae630 0x7f4be00aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 -- 172.21.15.92:0/3566581802 >> 172.21.15.92:0/3566581802 conn(0x7f4be0098c10 msgr2=0x7f4be00cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 -- 172.21.15.92:0/3566581802 shutdown_connections 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.702+0000 7f4be83a0700 1 -- 172.21.15.92:0/3566581802 wait complete. 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.703+0000 7f4be83a0700 1 Processor -- start 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.703+0000 7f4be83a0700 1 -- start start 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ac400 0x7f4be00c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4be00ae630 0x7f4be00c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00c6e60 con 0x7f4be00b1360 2023-06-09T04:35:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00b39c0 con 0x7f4be00ae630 2023-06-09T04:35:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00b3ca0 con 0x7f4be00ac400 2023-06-09T04:35:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38098/0 (socket says 172.21.15.92:38098) 2023-06-09T04:35:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 -- 172.21.15.92:0/3574589089 learned_addr learned my addr 172.21.15.92:0/3574589089 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be593b700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4be00ae630 0x7f4be00c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be613c700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ac400 0x7f4be00c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 -- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ac400 msgr2=0x7f4be00c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ac400 0x7f4be00c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 -- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4be00ae630 msgr2=0x7f4be00c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4be00ae630 0x7f4be00c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 -- 172.21.15.92:0/3574589089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd4009430 con 0x7f4be00b1360 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be693d700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00c37e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800cca0 tx=0x7f4bd800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3574589089 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800cea0 con 0x7f4be00b1360 2023-06-09T04:35:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3574589089 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd800ade0 con 0x7f4be00b1360 2023-06-09T04:35:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be00b3f80 con 0x7f4be00b1360 2023-06-09T04:35:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3574589089 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8007310 con 0x7f4be00b1360 2023-06-09T04:35:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.704+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be00c70c0 con 0x7f4be00b1360 2023-06-09T04:35:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.705+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be00740e0 con 0x7f4be00b1360 2023-06-09T04:35:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.706+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3574589089 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4bd8007470 con 0x7f4be00b1360 2023-06-09T04:35:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.706+0000 7f4bd2ffd700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4bc406ace0 0x7f4bc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.706+0000 7f4be613c700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4bc406ace0 0x7f4bc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.706+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3574589089 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4bd80893e0 con 0x7f4be00b1360 2023-06-09T04:35:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.707+0000 7f4be613c700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4bc406ace0 0x7f4bc406d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4013040 tx=0x7f4bd4008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.710+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3574589089 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4bd8054480 con 0x7f4be00b1360 2023-06-09T04:35:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.940+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be006e5f0 con 0x7f4be00b1360 2023-06-09T04:35:28.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.941+0000 7f4bd2ffd700 1 -- 172.21.15.92:0/3574589089 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f4bd8057aa0 con 0x7f4be00b1360 2023-06-09T04:35:28.941 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:28.942 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:35:28.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4bc406ace0 msgr2=0x7f4bc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4bc406ace0 0x7f4bc406d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4013040 tx=0x7f4bd4008070 comp rx=0 tx=0).stop 2023-06-09T04:35:28.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 msgr2=0x7f4be00c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:28.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00c37e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800cca0 tx=0x7f4bd800a340 comp rx=0 tx=0).stop 2023-06-09T04:35:28.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 shutdown_connections 2023-06-09T04:35:28.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4bc406ace0 0x7f4bc406d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4be00b1360 0x7f4be00c37e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4be00ac400 0x7f4be00c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 --2- 172.21.15.92:0/3574589089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4be00ae630 0x7f4be00c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:28.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 >> 172.21.15.92:0/3574589089 conn(0x7f4be0098c10 msgr2=0x7f4be0152e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:28.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 shutdown_connections 2023-06-09T04:35:28.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:28.943+0000 7f4be83a0700 1 -- 172.21.15.92:0/3574589089 wait complete. 2023-06-09T04:35:28.945 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:35:28.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d51535700 1 Processor -- start 2023-06-09T04:35:29.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d51535700 1 -- start start 2023-06-09T04:35:29.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d51535700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d4c0b45e0 0x7f5d4c0b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d51535700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d4c06bbd0 0x7f5d4c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d51535700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c0cd8a0 con 0x7f5d4c1603f0 2023-06-09T04:35:29.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d51535700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c0cf3d0 con 0x7f5d4c06bbd0 2023-06-09T04:35:29.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d51535700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c0b4ef0 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d4a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d4c06bbd0 0x7f5d4c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d4a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d4c06bbd0 0x7f5d4c15feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38102/0 (socket says 172.21.15.92:38102) 2023-06-09T04:35:29.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.076+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/4067204135 learned_addr learned my addr 172.21.15.92:0/4067204135 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:29.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d4affd700 1 --2- 172.21.15.92:0/4067204135 >> v2:172.21.15.123:3300/0 conn(0x7f5d4c0b45e0 0x7f5d4c0b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/4067204135 >> v2:172.21.15.123:3300/0 conn(0x7f5d4c0b45e0 msgr2=0x7f5d4c0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:29.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d4a7fc700 1 --2- 172.21.15.92:0/4067204135 >> v2:172.21.15.123:3300/0 conn(0x7f5d4c0b45e0 0x7f5d4c0b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/4067204135 >> v1:172.21.15.123:6789/0 conn(0x7f5d4c1603f0 legacy=0x7f5d4c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:29.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/4067204135 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0cf0f0 con 0x7f5d4c06bbd0 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d4a7fc700 1 --2- 172.21.15.92:0/4067204135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c06bbd0 0x7f5d4c15feb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c06ad80 tx=0x7f5d3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ed89ff6e4a6b2f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d49ffb700 1 -- 172.21.15.92:0/4067204135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004230 con 0x7f5d4c06bbd0 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d49ffb700 1 -- 172.21.15.92:0/4067204135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c004390 con 0x7f5d4c06bbd0 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d49ffb700 1 -- 172.21.15.92:0/4067204135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0036a0 con 0x7f5d4c06bbd0 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d51535700 1 -- 172.21.15.92:0/4067204135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c06bbd0 msgr2=0x7f5d4c15feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.077+0000 7f5d51535700 1 --2- 172.21.15.92:0/4067204135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c06bbd0 0x7f5d4c15feb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c06ad80 tx=0x7f5d3c009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.078+0000 7f5d51535700 1 -- 172.21.15.92:0/4067204135 shutdown_connections 2023-06-09T04:35:29.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.078+0000 7f5d51535700 1 --2- 172.21.15.92:0/4067204135 >> v2:172.21.15.123:3300/0 conn(0x7f5d4c0b45e0 0x7f5d4c0b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.078+0000 7f5d51535700 1 --2- 172.21.15.92:0/4067204135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c06bbd0 0x7f5d4c15feb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.078+0000 7f5d51535700 1 -- 172.21.15.92:0/4067204135 >> 172.21.15.92:0/4067204135 conn(0x7f5d4c0cf5e0 msgr2=0x7f5d4c0cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:29.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.078+0000 7f5d51535700 1 -- 172.21.15.92:0/4067204135 shutdown_connections 2023-06-09T04:35:29.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.078+0000 7f5d51535700 1 -- 172.21.15.92:0/4067204135 wait complete. 2023-06-09T04:35:29.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.079+0000 7f5d51535700 1 Processor -- start 2023-06-09T04:35:29.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.079+0000 7f5d51535700 1 -- start start 2023-06-09T04:35:29.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d4c06bbd0 0x7f5d4c1f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c0b45e0 0x7f5d4c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d4c1603f0 0x7f5d4c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c15a690 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c15a970 con 0x7f5d4c06bbd0 2023-06-09T04:35:29.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c15ac50 con 0x7f5d4c1603f0 2023-06-09T04:35:29.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c0b45e0 0x7f5d4c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c0b45e0 0x7f5d4c159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38106/0 (socket says 172.21.15.92:38106) 2023-06-09T04:35:29.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/3276129309 learned_addr learned my addr 172.21.15.92:0/3276129309 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4affd700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d4c06bbd0 0x7f5d4c1f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4b7fe700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d4c1603f0 0x7f5d4c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d4c1603f0 msgr2=0x7f5d4c0b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d4c1603f0 0x7f5d4c0b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d4c06bbd0 msgr2=0x7f5d4c1f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d4c06bbd0 0x7f5d4c1f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 -- 172.21.15.92:0/3276129309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c009550 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d4a7fc700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c0b45e0 0x7f5d4c159cb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009130 tx=0x7f5d3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d337fe700 1 -- 172.21.15.92:0/3276129309 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c011630 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d337fe700 1 -- 172.21.15.92:0/3276129309 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c011790 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c15af30 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d337fe700 1 -- 172.21.15.92:0/3276129309 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c011b00 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.080+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c1f5c10 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.081+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c1f5da0 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.082+0000 7f5d337fe700 1 -- 172.21.15.92:0/3276129309 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d3c011c60 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.082+0000 7f5d337fe700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5d3806ace0 0x7f5d3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.082+0000 7f5d337fe700 1 -- 172.21.15.92:0/3276129309 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d3c08ab50 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.082+0000 7f5d4affd700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5d3806ace0 0x7f5d3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.083+0000 7f5d4affd700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5d3806ace0 0x7f5d3806d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d40008850 tx=0x7f5d40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.086+0000 7f5d337fe700 1 -- 172.21.15.92:0/3276129309 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d3c055c90 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.272+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d4c15b450 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.274+0000 7f5d337fe700 1 -- 172.21.15.92:0/3276129309 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f5d4c15b450 con 0x7f5d4c0b45e0 2023-06-09T04:35:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:29.276 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:29.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5d3806ace0 msgr2=0x7f5d3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5d3806ace0 0x7f5d3806d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d40008850 tx=0x7f5d40008040 comp rx=0 tx=0).stop 2023-06-09T04:35:29.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c0b45e0 msgr2=0x7f5d4c159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c0b45e0 0x7f5d4c159cb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009130 tx=0x7f5d3c004750 comp rx=0 tx=0).stop 2023-06-09T04:35:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 shutdown_connections 2023-06-09T04:35:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5d3806ace0 0x7f5d3806d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5d4c1603f0 0x7f5d4c0b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d4c06bbd0 0x7f5d4c1f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 --2- 172.21.15.92:0/3276129309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d4c0b45e0 0x7f5d4c159cb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 >> 172.21.15.92:0/3276129309 conn(0x7f5d4c0cf5e0 msgr2=0x7f5d4c1e5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:29.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 shutdown_connections 2023-06-09T04:35:29.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.276+0000 7f5d51535700 1 -- 172.21.15.92:0/3276129309 wait complete. 2023-06-09T04:35:29.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:29.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:29.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.410+0000 7fbc05134700 1 Processor -- start 2023-06-09T04:35:29.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.410+0000 7fbc05134700 1 -- start start 2023-06-09T04:35:29.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbc05134700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc0006a4c0 0x7fbc000cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbc05134700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc00071dc0 con 0x7fbc000aea90 2023-06-09T04:35:29.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbc05134700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc000cf030 con 0x7fbc000aee60 2023-06-09T04:35:29.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbc05134700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc0015edf0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbbff59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc0006a4c0 0x7fbc000cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbbff59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc0006a4c0 0x7fbc000cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38114/0 (socket says 172.21.15.92:38114) 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbbff59e700 1 -- 172.21.15.92:0/534201550 learned_addr learned my addr 172.21.15.92:0/534201550 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbbff59e700 1 -- 172.21.15.92:0/534201550 >> v1:172.21.15.123:6789/0 conn(0x7fbc000aee60 legacy=0x7fbc0006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbbff59e700 1 -- 172.21.15.92:0/534201550 >> v1:172.21.15.92:6789/0 conn(0x7fbc000aea90 legacy=0x7fbc00069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.411+0000 7fbbff59e700 1 -- 172.21.15.92:0/534201550 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc000cf2d0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbbff59e700 1 --2- 172.21.15.92:0/534201550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc000cce60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009e50 tx=0x7fbbf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb294a823b3ffee server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbbfdd9b700 1 -- 172.21.15.92:0/534201550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800e070 con 0x7fbc0006a4c0 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbbfdd9b700 1 -- 172.21.15.92:0/534201550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf800b9c0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbbfdd9b700 1 -- 172.21.15.92:0/534201550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800bd70 con 0x7fbc0006a4c0 2023-06-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbc05134700 1 -- 172.21.15.92:0/534201550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 msgr2=0x7fbc000cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbc05134700 1 --2- 172.21.15.92:0/534201550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc000cce60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009e50 tx=0x7fbbf800b330 comp rx=0 tx=0).stop 2023-06-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbc05134700 1 -- 172.21.15.92:0/534201550 shutdown_connections 2023-06-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbc05134700 1 --2- 172.21.15.92:0/534201550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc000cce60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbc05134700 1 -- 172.21.15.92:0/534201550 >> 172.21.15.92:0/534201550 conn(0x7fbc000cc380 msgr2=0x7fbc000cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbc05134700 1 -- 172.21.15.92:0/534201550 shutdown_connections 2023-06-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.412+0000 7fbc05134700 1 -- 172.21.15.92:0/534201550 wait complete. 2023-06-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.414+0000 7fbc05134700 1 Processor -- start 2023-06-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.414+0000 7fbc05134700 1 -- start start 2023-06-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbc05134700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc001470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbc05134700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc000aea90 0x7fbc0015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbc05134700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc000c4480 0x7fbc0015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbc05134700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000c49d0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbc05134700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000c4cb0 con 0x7fbc000aea90 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbc05134700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000c4f90 con 0x7fbc000c4480 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc001470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc001470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38118/0 (socket says 172.21.15.92:38118) 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 -- 172.21.15.92:0/81647732 learned_addr learned my addr 172.21.15.92:0/81647732 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfe59c700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc000aea90 0x7fbc0015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 -- 172.21.15.92:0/81647732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc000c4480 msgr2=0x7fbc0015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc000c4480 0x7fbc0015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 -- 172.21.15.92:0/81647732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc000aea90 msgr2=0x7fbc0015d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc000aea90 0x7fbc0015d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.415+0000 7fbbfed9d700 1 -- 172.21.15.92:0/81647732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8009b30 con 0x7fbc0006a4c0 2023-06-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.416+0000 7fbbfed9d700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc001470a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400c4e0 tx=0x7fbbf400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.416+0000 7fbbe77fe700 1 -- 172.21.15.92:0/81647732 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40052d0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.416+0000 7fbbe77fe700 1 -- 172.21.15.92:0/81647732 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4005430 con 0x7fbc0006a4c0 2023-06-09T04:35:29.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.416+0000 7fbbe77fe700 1 -- 172.21.15.92:0/81647732 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf40038c0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.416+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc000c5270 con 0x7fbc0006a4c0 2023-06-09T04:35:29.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.416+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc001475e0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.417+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0004e8e0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.417+0000 7fbbe77fe700 1 -- 172.21.15.92:0/81647732 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbbf4003a20 con 0x7fbc0006a4c0 2023-06-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.417+0000 7fbbe77fe700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbec06ace0 0x7fbbec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.418+0000 7fbbe77fe700 1 -- 172.21.15.92:0/81647732 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbbf4088910 con 0x7fbc0006a4c0 2023-06-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.418+0000 7fbbfe59c700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbec06ace0 0x7fbbec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.418+0000 7fbbfe59c700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbec06ace0 0x7fbbec06d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc001f35c0 tx=0x7fbbf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.422+0000 7fbbe77fe700 1 -- 172.21.15.92:0/81647732 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbbf4053a50 con 0x7fbc0006a4c0 2023-06-09T04:35:29.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.607+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc0006a9b0 con 0x7fbc0006a4c0 2023-06-09T04:35:29.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.607+0000 7fbbe77fe700 1 -- 172.21.15.92:0/81647732 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbf4017020 con 0x7fbc0006a4c0 2023-06-09T04:35:29.609 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbec06ace0 msgr2=0x7fbbec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbec06ace0 0x7fbbec06d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc001f35c0 tx=0x7fbbf0009040 comp rx=0 tx=0).stop 2023-06-09T04:35:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 msgr2=0x7fbc001470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc001470a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400c4e0 tx=0x7fbbf400c8a0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 shutdown_connections 2023-06-09T04:35:29.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbec06ace0 0x7fbbec06d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc000c4480 0x7fbc0015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc0006a4c0 0x7fbc001470a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 --2- 172.21.15.92:0/81647732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc000aea90 0x7fbc0015d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 >> 172.21.15.92:0/81647732 conn(0x7fbc000cc380 msgr2=0x7fbc000cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:29.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.609+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 shutdown_connections 2023-06-09T04:35:29.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.610+0000 7fbc05134700 1 -- 172.21.15.92:0/81647732 wait complete. 2023-06-09T04:35:29.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.744+0000 7fcb2dd59700 1 Processor -- start 2023-06-09T04:35:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.744+0000 7fcb2dd59700 1 -- start start 2023-06-09T04:35:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.744+0000 7fcb2dd59700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 0x7fcb280713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.744+0000 7fcb2dd59700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb280ce190 con 0x7fcb280ce3d0 2023-06-09T04:35:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.744+0000 7fcb2dd59700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb28071ab0 con 0x7fcb2815d470 2023-06-09T04:35:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.744+0000 7fcb2dd59700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb28072350 con 0x7fcb280aed50 2023-06-09T04:35:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.744+0000 7fcb277fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcb280ce3d0 0x7fcb280ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36458/0 (socket says 172.21.15.92:36458) 2023-06-09T04:35:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb277fe700 1 -- 172.21.15.92:0/3327429716 learned_addr learned my addr 172.21.15.92:0/3327429716 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 0x7fcb280713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745235422 0 0) 0x7fcb280ce190 con 0x7fcb280ce3d0 2023-06-09T04:35:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb140035b0 con 0x7fcb280ce3d0 2023-06-09T04:35:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827477791 0 0) 0x7fcb28071ab0 con 0x7fcb2815d470 2023-06-09T04:35:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb280ce190 con 0x7fcb2815d470 2023-06-09T04:35:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550361738 0 0) 0x7fcb140035b0 con 0x7fcb280ce3d0 2023-06-09T04:35:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 msgr2=0x7fcb280713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 0x7fcb280713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 >> v1:172.21.15.123:6789/0 conn(0x7fcb2815d470 legacy=0x7fcb281596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28071d90 con 0x7fcb280ce3d0 2023-06-09T04:35:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fcb18001670 con 0x7fcb280ce3d0 2023-06-09T04:35:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 >> v1:172.21.15.92:6789/0 conn(0x7fcb280ce3d0 legacy=0x7fcb280ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 0x7fcb14003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14004120 0x7fcb14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14004d90 0x7fcb14005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb28072350 con 0x7fcb14004120 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1400dcd0 con 0x7fcb14004d90 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1400de90 con 0x7fcb280aed50 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14004120 0x7fcb14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb277fe700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 0x7fcb14003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb27fff700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14004d90 0x7fcb14005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb26ffd700 1 -- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 msgr2=0x7fcb14003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb280aed50 0x7fcb14003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb26ffd700 1 -- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14004d90 msgr2=0x7fcb14005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb14004d90 0x7fcb14005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.745+0000 7fcb26ffd700 1 -- 172.21.15.92:0/3327429716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140035b0 con 0x7fcb14004120 2023-06-09T04:35:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14004120 0x7fcb14004850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009b30 tx=0x7fcb14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5695fd3dc92a1c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000e6c0 con 0x7fcb14004120 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1000e820 con 0x7fcb14004120 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb267fc700 1 -- 172.21.15.92:0/3327429716 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000eb90 con 0x7fcb14004120 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3327429716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14004120 msgr2=0x7fcb14004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb2dd59700 1 --2- 172.21.15.92:0/3327429716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb14004120 0x7fcb14004850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009b30 tx=0x7fcb14005880 comp rx=0 tx=0).stop 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb27fff700 1 -- 172.21.15.92:0/3327429716 reap_dead start 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3327429716 shutdown_connections 2023-06-09T04:35:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3327429716 >> 172.21.15.92:0/3327429716 conn(0x7fcb280cc380 msgr2=0x7fcb280cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3327429716 shutdown_connections 2023-06-09T04:35:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.746+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3327429716 wait complete. 2023-06-09T04:35:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.748+0000 7fcb2dd59700 1 Processor -- start 2023-06-09T04:35:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.748+0000 7fcb2dd59700 1 -- start start 2023-06-09T04:35:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb2dd59700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb14004120 0x7fcb28152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb2dd59700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb281511d0 0x7fcb281556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb2dd59700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb281f1b40 0x7fcb281f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb2dd59700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb281567c0 con 0x7fcb281511d0 2023-06-09T04:35:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb2dd59700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb281f2500 con 0x7fcb281f1b40 2023-06-09T04:35:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb2dd59700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb280c7a10 con 0x7fcb14004120 2023-06-09T04:35:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb281511d0 0x7fcb281556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb281511d0 0x7fcb281556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38140/0 (socket says 172.21.15.92:38140) 2023-06-09T04:35:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 -- 172.21.15.92:0/3692547547 learned_addr learned my addr 172.21.15.92:0/3692547547 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb277fe700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb14004120 0x7fcb28152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb27fff700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb281f1b40 0x7fcb281f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 -- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb14004120 msgr2=0x7fcb28152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb14004120 0x7fcb28152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 -- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb281f1b40 msgr2=0x7fcb281f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb281f1b40 0x7fcb281f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb26ffd700 1 -- 172.21.15.92:0/3692547547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28155f20 con 0x7fcb281511d0 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.749+0000 7fcb27fff700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb281f1b40 0x7fcb281f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.750+0000 7fcb26ffd700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb281511d0 0x7fcb281556c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb100090b0 tx=0x7fcb100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.750+0000 7fcb1ffff700 1 -- 172.21.15.92:0/3692547547 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10021070 con 0x7fcb281511d0 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.750+0000 7fcb1ffff700 1 -- 172.21.15.92:0/3692547547 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10004550 con 0x7fcb281511d0 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.750+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb28156200 con 0x7fcb281511d0 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.750+0000 7fcb1ffff700 1 -- 172.21.15.92:0/3692547547 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000e6c0 con 0x7fcb281511d0 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.750+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb280c7cb0 con 0x7fcb281511d0 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.751+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb280c7e40 con 0x7fcb281511d0 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.751+0000 7fcb1ffff700 1 -- 172.21.15.92:0/3692547547 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb10025070 con 0x7fcb281511d0 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.751+0000 7fcb1ffff700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcb0806ace0 0x7fcb0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.751+0000 7fcb277fe700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcb0806ace0 0x7fcb0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.751+0000 7fcb1ffff700 1 -- 172.21.15.92:0/3692547547 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb1008a620 con 0x7fcb281511d0 2023-06-09T04:35:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.752+0000 7fcb277fe700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcb0806ace0 0x7fcb0806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000ff0 tx=0x7fcb18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.756+0000 7fcb1ffff700 1 -- 172.21.15.92:0/3692547547 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb10055880 con 0x7fcb281511d0 2023-06-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.944+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcb28156720 con 0x7fcb281511d0 2023-06-09T04:35:29.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.944+0000 7fcb1ffff700 1 -- 172.21.15.92:0/3692547547 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcb10058ea0 con 0x7fcb281511d0 2023-06-09T04:35:29.946 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.946+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcb0806ace0 msgr2=0x7fcb0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.946+0000 7fcb2dd59700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcb0806ace0 0x7fcb0806d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb18000ff0 tx=0x7fcb18009040 comp rx=0 tx=0).stop 2023-06-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.946+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb281511d0 msgr2=0x7fcb281556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.946+0000 7fcb2dd59700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb281511d0 0x7fcb281556c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb100090b0 tx=0x7fcb100047c0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.946+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 shutdown_connections 2023-06-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.947+0000 7fcb2dd59700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcb0806ace0 0x7fcb0806d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.947+0000 7fcb2dd59700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb281f1b40 0x7fcb281f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.947+0000 7fcb2dd59700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcb14004120 0x7fcb28152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.947+0000 7fcb2dd59700 1 --2- 172.21.15.92:0/3692547547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb281511d0 0x7fcb281556c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.947+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 >> 172.21.15.92:0/3692547547 conn(0x7fcb280cc380 msgr2=0x7fcb28158920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.947+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 shutdown_connections 2023-06-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:29.947+0000 7fcb2dd59700 1 -- 172.21.15.92:0/3692547547 wait complete. 2023-06-09T04:35:29.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.081+0000 7f3a5bc40700 1 Processor -- start 2023-06-09T04:35:30.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.081+0000 7f3a5bc40700 1 -- start start 2023-06-09T04:35:30.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a5bc40700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a540b60d0 0x7f3a540d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a5bc40700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a5415e810 0x7f3a540cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a5bc40700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a540cb700 con 0x7f3a540cd8c0 2023-06-09T04:35:30.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a5bc40700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a540cdd40 con 0x7f3a540b60d0 2023-06-09T04:35:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a5bc40700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a540b64a0 con 0x7f3a5415e810 2023-06-09T04:35:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a599dc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a540b60d0 0x7f3a540d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a599dc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3a540b60d0 0x7f3a540d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38146/0 (socket says 172.21.15.92:38146) 2023-06-09T04:35:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.082+0000 7f3a599dc700 1 -- 172.21.15.92:0/3394995168 learned_addr learned my addr 172.21.15.92:0/3394995168 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a591db700 1 --2- 172.21.15.92:0/3394995168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a5415e810 0x7f3a540cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a599dc700 1 -- 172.21.15.92:0/3394995168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a5415e810 msgr2=0x7f3a540cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a599dc700 1 --2- 172.21.15.92:0/3394995168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a5415e810 0x7f3a540cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a599dc700 1 -- 172.21.15.92:0/3394995168 >> v1:172.21.15.123:6789/0 conn(0x7f3a540cd8c0 legacy=0x7f3a540ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a599dc700 1 -- 172.21.15.92:0/3394995168 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a540ccf50 con 0x7f3a540b60d0 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a599dc700 1 --2- 172.21.15.92:0/3394995168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a540b60d0 0x7f3a540d03b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a440098f0 tx=0x7f3a44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5141d2742bba922 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a589da700 1 -- 172.21.15.92:0/3394995168 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44004230 con 0x7f3a540b60d0 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a589da700 1 -- 172.21.15.92:0/3394995168 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a44004390 con 0x7f3a540b60d0 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a589da700 1 -- 172.21.15.92:0/3394995168 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a440036a0 con 0x7f3a540b60d0 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3394995168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a540b60d0 msgr2=0x7f3a540d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3394995168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a540b60d0 0x7f3a540d03b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a440098f0 tx=0x7f3a44009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.083+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3394995168 shutdown_connections 2023-06-09T04:35:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.084+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3394995168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a540b60d0 0x7f3a540d03b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.084+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3394995168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a5415e810 0x7f3a540cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.084+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3394995168 >> 172.21.15.92:0/3394995168 conn(0x7f3a5406e4d0 msgr2=0x7f3a5406e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.084+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3394995168 shutdown_connections 2023-06-09T04:35:30.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.084+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3394995168 wait complete. 2023-06-09T04:35:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.085+0000 7f3a5bc40700 1 Processor -- start 2023-06-09T04:35:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.085+0000 7f3a5bc40700 1 -- start start 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.085+0000 7f3a5bc40700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a540b60d0 0x7f3a54153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.085+0000 7f3a5bc40700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a540cd8c0 0x7f3a540c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.085+0000 7f3a5bc40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a5415e810 0x7f3a54155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5bc40700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a541e4f60 con 0x7f3a5415e810 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5bc40700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54155ec0 con 0x7f3a540b60d0 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5bc40700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54154c60 con 0x7f3a540cd8c0 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a5415e810 0x7f3a54155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a5415e810 0x7f3a54155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38156/0 (socket says 172.21.15.92:38156) 2023-06-09T04:35:30.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 -- 172.21.15.92:0/3125796496 learned_addr learned my addr 172.21.15.92:0/3125796496 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:30.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a591db700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a540cd8c0 0x7f3a540c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a599dc700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a540b60d0 0x7f3a54153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 -- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a540cd8c0 msgr2=0x7f3a540c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a540cd8c0 0x7f3a540c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 -- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a540b60d0 msgr2=0x7f3a54153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a540b60d0 0x7f3a54153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 -- 172.21.15.92:0/3125796496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44009550 con 0x7f3a5415e810 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5a1dd700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a5415e810 0x7f3a54155980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00b970 tx=0x7f3a4c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a4a7fc700 1 -- 172.21.15.92:0/3125796496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00f860 con 0x7f3a5415e810 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a4a7fc700 1 -- 172.21.15.92:0/3125796496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c00f9c0 con 0x7f3a5415e810 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a541543c0 con 0x7f3a5415e810 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a4a7fc700 1 -- 172.21.15.92:0/3125796496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00fd70 con 0x7f3a5415e810 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.086+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a54154980 con 0x7f3a5415e810 2023-06-09T04:35:30.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.087+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a541e4cc0 con 0x7f3a5415e810 2023-06-09T04:35:30.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.087+0000 7f3a4a7fc700 1 -- 172.21.15.92:0/3125796496 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a4c0203f0 con 0x7f3a5415e810 2023-06-09T04:35:30.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.088+0000 7f3a4a7fc700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a3806ace0 0x7f3a3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.088+0000 7f3a4a7fc700 1 -- 172.21.15.92:0/3125796496 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a4c0897d0 con 0x7f3a5415e810 2023-06-09T04:35:30.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.088+0000 7f3a599dc700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a3806ace0 0x7f3a3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.088+0000 7f3a599dc700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a3806ace0 0x7f3a3806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009130 tx=0x7f3a44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.092+0000 7f3a4a7fc700 1 -- 172.21.15.92:0/3125796496 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a541e4cc0 con 0x7f3a5415e810 2023-06-09T04:35:30.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.281+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a541e50a0 con 0x7f3a5415e810 2023-06-09T04:35:30.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.281+0000 7f3a4a7fc700 1 -- 172.21.15.92:0/3125796496 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a541e50a0 con 0x7f3a5415e810 2023-06-09T04:35:30.283 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:30.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a3806ace0 msgr2=0x7f3a3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a3806ace0 0x7f3a3806d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009130 tx=0x7f3a44008030 comp rx=0 tx=0).stop 2023-06-09T04:35:30.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a5415e810 msgr2=0x7f3a54155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a5415e810 0x7f3a54155980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00b970 tx=0x7f3a4c00bc80 comp rx=0 tx=0).stop 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 shutdown_connections 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3a3806ace0 0x7f3a3806d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a5415e810 0x7f3a54155980 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a540b60d0 0x7f3a54153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.283+0000 7f3a5bc40700 1 --2- 172.21.15.92:0/3125796496 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3a540cd8c0 0x7f3a540c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.284+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 >> 172.21.15.92:0/3125796496 conn(0x7f3a5406e4d0 msgr2=0x7f3a540cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.284+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 shutdown_connections 2023-06-09T04:35:30.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.284+0000 7f3a5bc40700 1 -- 172.21.15.92:0/3125796496 wait complete. 2023-06-09T04:35:30.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:30.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.418+0000 7f77acd6c700 1 Processor -- start 2023-06-09T04:35:30.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.418+0000 7f77acd6c700 1 -- start start 2023-06-09T04:35:30.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77acd6c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77a8069db0 0x7f77a815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77acd6c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f77a80cce60 0x7f77a806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77acd6c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a815d040 con 0x7f77a80cf440 2023-06-09T04:35:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77acd6c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a8160ab0 con 0x7f77a8069db0 2023-06-09T04:35:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77acd6c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a815ca10 con 0x7f77a80cce60 2023-06-09T04:35:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77a8069db0 0x7f77a815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f77a8069db0 0x7f77a815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38160/0 (socket says 172.21.15.92:38160) 2023-06-09T04:35:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 -- 172.21.15.92:0/4203601641 learned_addr learned my addr 172.21.15.92:0/4203601641 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a6d9d700 1 --2- 172.21.15.92:0/4203601641 >> v2:172.21.15.123:3300/0 conn(0x7f77a80cce60 0x7f77a806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 -- 172.21.15.92:0/4203601641 >> v2:172.21.15.123:3300/0 conn(0x7f77a80cce60 msgr2=0x7f77a806ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 --2- 172.21.15.92:0/4203601641 >> v2:172.21.15.123:3300/0 conn(0x7f77a80cce60 0x7f77a806ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 -- 172.21.15.92:0/4203601641 >> v1:172.21.15.123:6789/0 conn(0x7f77a80cf440 legacy=0x7f77a806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 -- 172.21.15.92:0/4203601641 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a8160d50 con 0x7f77a8069db0 2023-06-09T04:35:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a6d9d700 1 --2- 172.21.15.92:0/4203601641 >> v2:172.21.15.123:3300/0 conn(0x7f77a80cce60 0x7f77a806ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a659c700 1 --2- 172.21.15.92:0/4203601641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a8069db0 0x7f77a815d4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77a8071340 tx=0x7f7798009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a36cf3a7fe42922 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a559a700 1 -- 172.21.15.92:0/4203601641 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798004230 con 0x7f77a8069db0 2023-06-09T04:35:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a559a700 1 -- 172.21.15.92:0/4203601641 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798004390 con 0x7f77a8069db0 2023-06-09T04:35:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.419+0000 7f77a559a700 1 -- 172.21.15.92:0/4203601641 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77980036a0 con 0x7f77a8069db0 2023-06-09T04:35:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 -- 172.21.15.92:0/4203601641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a8069db0 msgr2=0x7f77a815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 --2- 172.21.15.92:0/4203601641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a8069db0 0x7f77a815d4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77a8071340 tx=0x7f7798009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 -- 172.21.15.92:0/4203601641 shutdown_connections 2023-06-09T04:35:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 --2- 172.21.15.92:0/4203601641 >> v2:172.21.15.123:3300/0 conn(0x7f77a80cce60 0x7f77a806ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 --2- 172.21.15.92:0/4203601641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a8069db0 0x7f77a815d4d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 -- 172.21.15.92:0/4203601641 >> 172.21.15.92:0/4203601641 conn(0x7f77a80cc380 msgr2=0x7f77a80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 -- 172.21.15.92:0/4203601641 shutdown_connections 2023-06-09T04:35:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.420+0000 7f77acd6c700 1 -- 172.21.15.92:0/4203601641 wait complete. 2023-06-09T04:35:30.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.421+0000 7f77acd6c700 1 Processor -- start 2023-06-09T04:35:30.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77acd6c700 1 -- start start 2023-06-09T04:35:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77acd6c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a8069db0 0x7f77a80c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77acd6c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a80cce60 0x7f77a8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77acd6c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80cf440 0x7f77a80c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77acd6c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80af470 con 0x7f77a80cf440 2023-06-09T04:35:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77acd6c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80af750 con 0x7f77a80cce60 2023-06-09T04:35:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77acd6c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80afa30 con 0x7f77a8069db0 2023-06-09T04:35:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77a6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80cf440 0x7f77a80c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77a6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80cf440 0x7f77a80c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38168/0 (socket says 172.21.15.92:38168) 2023-06-09T04:35:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.422+0000 7f77a6d9d700 1 -- 172.21.15.92:0/510435919 learned_addr learned my addr 172.21.15.92:0/510435919 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a659c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a8069db0 0x7f77a80c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a5d9b700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a80cce60 0x7f77a8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a6d9d700 1 -- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a8069db0 msgr2=0x7f77a80c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a6d9d700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a8069db0 0x7f77a80c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a6d9d700 1 -- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a80cce60 msgr2=0x7f77a8155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a6d9d700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a80cce60 0x7f77a8155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a6d9d700 1 -- 172.21.15.92:0/510435919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798009550 con 0x7f77a80cf440 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a5d9b700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a80cce60 0x7f77a8155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77a6d9d700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80cf440 0x7f77a80c8ba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77a000cef0 tx=0x7f77a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f7796ffd700 1 -- 172.21.15.92:0/510435919 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000ae50 con 0x7f77a80cf440 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f7796ffd700 1 -- 172.21.15.92:0/510435919 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a0015440 con 0x7f77a80cf440 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a80afd10 con 0x7f77a80cf440 2023-06-09T04:35:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f7796ffd700 1 -- 172.21.15.92:0/510435919 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a00157f0 con 0x7f77a80cf440 2023-06-09T04:35:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.423+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a80c61b0 con 0x7f77a80cf440 2023-06-09T04:35:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.424+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a80b0230 con 0x7f77a80cf440 2023-06-09T04:35:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.424+0000 7f7796ffd700 1 -- 172.21.15.92:0/510435919 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f77a000d430 con 0x7f77a80cf440 2023-06-09T04:35:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.425+0000 7f7796ffd700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f778806ace0 0x7f778806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.425+0000 7f7796ffd700 1 -- 172.21.15.92:0/510435919 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77a0089a50 con 0x7f77a80cf440 2023-06-09T04:35:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.425+0000 7f77a659c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f778806ace0 0x7f778806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.425+0000 7f77a659c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f778806ace0 0x7f778806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7798009130 tx=0x7f7798008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.429+0000 7f7796ffd700 1 -- 172.21.15.92:0/510435919 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f77a0054b90 con 0x7f77a80cf440 2023-06-09T04:35:30.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.615+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77a806be50 con 0x7f77a80cf440 2023-06-09T04:35:30.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.616+0000 7f7796ffd700 1 -- 172.21.15.92:0/510435919 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77a001a0a0 con 0x7f77a80cf440 2023-06-09T04:35:30.618 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:30.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f778806ace0 msgr2=0x7f778806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f778806ace0 0x7f778806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7798009130 tx=0x7f7798008030 comp rx=0 tx=0).stop 2023-06-09T04:35:30.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80cf440 msgr2=0x7f77a80c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80cf440 0x7f77a80c8ba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77a000cef0 tx=0x7f77a000a340 comp rx=0 tx=0).stop 2023-06-09T04:35:30.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 shutdown_connections 2023-06-09T04:35:30.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f778806ace0 0x7f778806d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77a80cf440 0x7f77a80c8ba0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77a8069db0 0x7f77a80c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 --2- 172.21.15.92:0/510435919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77a80cce60 0x7f77a8155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 >> 172.21.15.92:0/510435919 conn(0x7f77a80cc380 msgr2=0x7f77a80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:30.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 shutdown_connections 2023-06-09T04:35:30.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.618+0000 7f77acd6c700 1 -- 172.21.15.92:0/510435919 wait complete. 2023-06-09T04:35:30.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:35:30.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.749+0000 7f8d61a07700 1 Processor -- start 2023-06-09T04:35:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.749+0000 7f8d61a07700 1 -- start start 2023-06-09T04:35:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d61a07700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d5c0aead0 0x7f8d5c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d61a07700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d5c15b380 0x7f8d5c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d61a07700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c071dd0 con 0x7f8d5c0aeea0 2023-06-09T04:35:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d61a07700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5c072300 con 0x7f8d5c15b380 2023-06-09T04:35:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d61a07700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5c0cf140 con 0x7f8d5c0aead0 2023-06-09T04:35:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d5c15b380 0x7f8d5c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d5c15b380 0x7f8d5c069db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38178/0 (socket says 172.21.15.92:38178) 2023-06-09T04:35:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5b7fe700 1 -- 172.21.15.92:0/3172999250 learned_addr learned my addr 172.21.15.92:0/3172999250 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5affd700 1 --2- 172.21.15.92:0/3172999250 >> v2:172.21.15.123:3300/0 conn(0x7f8d5c0aead0 0x7f8d5c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5b7fe700 1 -- 172.21.15.92:0/3172999250 >> v2:172.21.15.123:3300/0 conn(0x7f8d5c0aead0 msgr2=0x7f8d5c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5b7fe700 1 --2- 172.21.15.92:0/3172999250 >> v2:172.21.15.123:3300/0 conn(0x7f8d5c0aead0 0x7f8d5c0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5b7fe700 1 -- 172.21.15.92:0/3172999250 >> v1:172.21.15.123:6789/0 conn(0x7f8d5c0aeea0 legacy=0x7f8d5c0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.750+0000 7f8d5b7fe700 1 -- 172.21.15.92:0/3172999250 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0cf420 con 0x7f8d5c15b380 2023-06-09T04:35:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d5b7fe700 1 --2- 172.21.15.92:0/3172999250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c15b380 0x7f8d5c069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c06ae70 tx=0x7f8d5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4571c7e94df52b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d59ffb700 1 -- 172.21.15.92:0/3172999250 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5400e070 con 0x7f8d5c15b380 2023-06-09T04:35:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d59ffb700 1 -- 172.21.15.92:0/3172999250 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5400b910 con 0x7f8d5c15b380 2023-06-09T04:35:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d59ffb700 1 -- 172.21.15.92:0/3172999250 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5400bcc0 con 0x7f8d5c15b380 2023-06-09T04:35:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 -- 172.21.15.92:0/3172999250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c15b380 msgr2=0x7f8d5c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 --2- 172.21.15.92:0/3172999250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c15b380 0x7f8d5c069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c06ae70 tx=0x7f8d5400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 -- 172.21.15.92:0/3172999250 shutdown_connections 2023-06-09T04:35:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 --2- 172.21.15.92:0/3172999250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c15b380 0x7f8d5c069db0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 --2- 172.21.15.92:0/3172999250 >> v2:172.21.15.123:3300/0 conn(0x7f8d5c0aead0 0x7f8d5c0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 -- 172.21.15.92:0/3172999250 >> 172.21.15.92:0/3172999250 conn(0x7f8d5c06aec0 msgr2=0x7f8d5c06b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 -- 172.21.15.92:0/3172999250 shutdown_connections 2023-06-09T04:35:30.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.751+0000 7f8d61a07700 1 -- 172.21.15.92:0/3172999250 wait complete. 2023-06-09T04:35:30.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.752+0000 7f8d61a07700 1 Processor -- start 2023-06-09T04:35:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d61a07700 1 -- start start 2023-06-09T04:35:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d61a07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c0aead0 0x7f8d5c151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d61a07700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d5c0aeea0 0x7f8d5c1f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d61a07700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d5c15b380 0x7f8d5c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d61a07700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0cae90 con 0x7f8d5c0aead0 2023-06-09T04:35:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d61a07700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0acaf0 con 0x7f8d5c0aeea0 2023-06-09T04:35:30.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d61a07700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c1f3050 con 0x7f8d5c15b380 2023-06-09T04:35:30.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.753+0000 7f8d5affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c0aead0 0x7f8d5c151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c0aead0 0x7f8d5c151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38180/0 (socket says 172.21.15.92:38180) 2023-06-09T04:35:30.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 -- 172.21.15.92:0/187296769 learned_addr learned my addr 172.21.15.92:0/187296769 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:30.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5a7fc700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d5c0aeea0 0x7f8d5c1f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5b7fe700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d5c15b380 0x7f8d5c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 -- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d5c15b380 msgr2=0x7f8d5c158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d5c15b380 0x7f8d5c158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 -- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d5c0aeea0 msgr2=0x7f8d5c1f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d5c0aeea0 0x7f8d5c1f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 -- 172.21.15.92:0/187296769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d54009b30 con 0x7f8d5c0aead0 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5b7fe700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d5c15b380 0x7f8d5c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d5affd700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c0aead0 0x7f8d5c151590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e9e0 tx=0x7f8d5000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d43fff700 1 -- 172.21.15.92:0/187296769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50009710 con 0x7f8d5c0aead0 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d43fff700 1 -- 172.21.15.92:0/187296769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5000c850 con 0x7f8d5c0aead0 2023-06-09T04:35:30.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d43fff700 1 -- 172.21.15.92:0/187296769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5000cbc0 con 0x7f8d5c0aead0 2023-06-09T04:35:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c0c3ea0 con 0x7f8d5c0aead0 2023-06-09T04:35:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.754+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c0ca5f0 con 0x7f8d5c0aead0 2023-06-09T04:35:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.755+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c0c41d0 con 0x7f8d5c0aead0 2023-06-09T04:35:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.756+0000 7f8d43fff700 1 -- 172.21.15.92:0/187296769 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d5000cd20 con 0x7f8d5c0aead0 2023-06-09T04:35:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.756+0000 7f8d43fff700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d4806ace0 0x7f8d4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:30.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.756+0000 7f8d43fff700 1 -- 172.21.15.92:0/187296769 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d50089510 con 0x7f8d5c0aead0 2023-06-09T04:35:30.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.756+0000 7f8d5a7fc700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d4806ace0 0x7f8d4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:30.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.756+0000 7f8d5a7fc700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d4806ace0 0x7f8d4806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0b2230 tx=0x7f8d4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:30.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.760+0000 7f8d43fff700 1 -- 172.21.15.92:0/187296769 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d500545c0 con 0x7f8d5c0aead0 2023-06-09T04:35:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.988+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d5c06be50 con 0x7f8d5c0aead0 2023-06-09T04:35:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.988+0000 7f8d43fff700 1 -- 172.21.15.92:0/187296769 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f8d50057be0 con 0x7f8d5c0aead0 2023-06-09T04:35:30.990 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:30.990 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:35:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d4806ace0 msgr2=0x7f8d4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d4806ace0 0x7f8d4806d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0b2230 tx=0x7f8d4c009040 comp rx=0 tx=0).stop 2023-06-09T04:35:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c0aead0 msgr2=0x7f8d5c151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c0aead0 0x7f8d5c151590 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e9e0 tx=0x7f8d5000ecf0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 shutdown_connections 2023-06-09T04:35:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d4806ace0 0x7f8d4806d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d5c15b380 0x7f8d5c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5c0aead0 0x7f8d5c151590 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 --2- 172.21.15.92:0/187296769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d5c0aeea0 0x7f8d5c1f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.990+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 >> 172.21.15.92:0/187296769 conn(0x7f8d5c06aec0 msgr2=0x7f8d5c15d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.991+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 shutdown_connections 2023-06-09T04:35:30.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:30.991+0000 7f8d61a07700 1 -- 172.21.15.92:0/187296769 wait complete. 2023-06-09T04:35:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:35:31.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:31.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.122+0000 7fc7f4515700 1 Processor -- start 2023-06-09T04:35:31.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.122+0000 7fc7f4515700 1 -- start start 2023-06-09T04:35:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f4515700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7ec15c7d0 0x7fc7ec0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f4515700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ac400 0x7fc7ec0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f4515700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7ec0ce730 con 0x7fc7ec0ae750 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f4515700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7ec06aaf0 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f4515700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec15cba0 con 0x7fc7ec0ac400 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f22b1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7ec15c7d0 0x7fc7ec0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f2ab2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7ec0ae750 0x7fc7ec0b1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36524/0 (socket says 172.21.15.92:36524) 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f2ab2700 1 -- 172.21.15.92:0/2006301521 learned_addr learned my addr 172.21.15.92:0/2006301521 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:31.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f22b1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc7ec15c7d0 0x7fc7ec0ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38188/0 (socket says 172.21.15.92:38188) 2023-06-09T04:35:31.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f1ab0700 1 --2- 172.21.15.92:0/2006301521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ac400 0x7fc7ec0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f22b1700 1 -- 172.21.15.92:0/2006301521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ac400 msgr2=0x7fc7ec0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:31.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f22b1700 1 --2- 172.21.15.92:0/2006301521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ac400 0x7fc7ec0ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f22b1700 1 -- 172.21.15.92:0/2006301521 >> v1:172.21.15.92:6789/0 conn(0x7fc7ec0ae750 legacy=0x7fc7ec0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f12af700 1 -- 172.21.15.92:0/2006301521 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149852528 0 0) 0x7fc7e4001490 con 0x7fc7ec0ae750 2023-06-09T04:35:31.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f22b1700 1 -- 172.21.15.92:0/2006301521 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec069ee0 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.123+0000 7fc7f22b1700 1 --2- 172.21.15.92:0/2006301521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0ce1f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009820 tx=0x7fc7e8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939bc896e2afa39a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:31.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f12af700 1 -- 172.21.15.92:0/2006301521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8005e30 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f12af700 1 -- 172.21.15.92:0/2006301521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e8006030 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f12af700 1 -- 172.21.15.92:0/2006301521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e80063c0 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 -- 172.21.15.92:0/2006301521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 msgr2=0x7fc7ec0ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 --2- 172.21.15.92:0/2006301521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0ce1f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009820 tx=0x7fc7e8009b30 comp rx=0 tx=0).stop 2023-06-09T04:35:31.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 -- 172.21.15.92:0/2006301521 shutdown_connections 2023-06-09T04:35:31.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 --2- 172.21.15.92:0/2006301521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0ce1f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 --2- 172.21.15.92:0/2006301521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ac400 0x7fc7ec0ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 -- 172.21.15.92:0/2006301521 >> 172.21.15.92:0/2006301521 conn(0x7fc7ec098c10 msgr2=0x7fc7ec0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:31.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 -- 172.21.15.92:0/2006301521 shutdown_connections 2023-06-09T04:35:31.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.124+0000 7fc7f4515700 1 -- 172.21.15.92:0/2006301521 wait complete. 2023-06-09T04:35:31.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.126+0000 7fc7f4515700 1 Processor -- start 2023-06-09T04:35:31.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.126+0000 7fc7f4515700 1 -- start start 2023-06-09T04:35:31.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec0ac400 0x7fc7ec1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ae750 0x7fc7ec155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec156290 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec0c5cd0 con 0x7fc7ec0ac400 2023-06-09T04:35:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec0c5fb0 con 0x7fc7ec0ae750 2023-06-09T04:35:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38198/0 (socket says 172.21.15.92:38198) 2023-06-09T04:35:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 -- 172.21.15.92:0/3520878912 learned_addr learned my addr 172.21.15.92:0/3520878912 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f1ab0700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ae750 0x7fc7ec155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f22b1700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec0ac400 0x7fc7ec1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 -- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ae750 msgr2=0x7fc7ec155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ae750 0x7fc7ec155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 -- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec0ac400 msgr2=0x7fc7ec1f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec0ac400 0x7fc7ec1f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 -- 172.21.15.92:0/3520878912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e8009430 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f22b1700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec0ac400 0x7fc7ec1f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f2ab2700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0c5360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400cb30 tx=0x7fc7e400ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7e2ffd700 1 -- 172.21.15.92:0/3520878912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400aaa0 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7e2ffd700 1 -- 172.21.15.92:0/3520878912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e400ac00 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7e2ffd700 1 -- 172.21.15.92:0/3520878912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40070d0 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ec1f1f00 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.127+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ec1f1b70 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.128+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ec1f2420 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.129+0000 7fc7e2ffd700 1 -- 172.21.15.92:0/3520878912 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7e4007230 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.129+0000 7fc7e2ffd700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc7d006ace0 0x7fc7d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.129+0000 7fc7f22b1700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc7d006ace0 0x7fc7d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.129+0000 7fc7e2ffd700 1 -- 172.21.15.92:0/3520878912 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7e4088c10 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.129+0000 7fc7f22b1700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc7d006ace0 0x7fc7d006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8000c00 tx=0x7fc7e8008070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:31.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.133+0000 7fc7e2ffd700 1 -- 172.21.15.92:0/3520878912 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc7ec1f2420 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.325+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7ec0ce730 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.326+0000 7fc7e2ffd700 1 -- 172.21.15.92:0/3520878912 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7fc7ec0ce730 con 0x7fc7ec15c7d0 2023-06-09T04:35:31.328 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:31.328 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:31.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.328+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc7d006ace0 msgr2=0x7fc7d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.328+0000 7fc7f4515700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc7d006ace0 0x7fc7d006d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8000c00 tx=0x7fc7e8008070 comp rx=0 tx=0).stop 2023-06-09T04:35:31.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.328+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 msgr2=0x7fc7ec0c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.328+0000 7fc7f4515700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0c5360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400cb30 tx=0x7fc7e400ce40 comp rx=0 tx=0).stop 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.328+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 shutdown_connections 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.328+0000 7fc7f4515700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc7d006ace0 0x7fc7d006d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.329+0000 7fc7f4515700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7ec15c7d0 0x7fc7ec0c5360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.329+0000 7fc7f4515700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec0ac400 0x7fc7ec1f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.329+0000 7fc7f4515700 1 --2- 172.21.15.92:0/3520878912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7ec0ae750 0x7fc7ec155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.329+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 >> 172.21.15.92:0/3520878912 conn(0x7fc7ec098c10 msgr2=0x7fc7ec0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.329+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 shutdown_connections 2023-06-09T04:35:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.329+0000 7fc7f4515700 1 -- 172.21.15.92:0/3520878912 wait complete. 2023-06-09T04:35:31.342 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:35:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.460+0000 7f3f75c53700 1 Processor -- start 2023-06-09T04:35:31.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.461+0000 7f3f75c53700 1 -- start start 2023-06-09T04:35:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.461+0000 7f3f75c53700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f700ae630 0x7f3f700aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.461+0000 7f3f75c53700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f700adde0 con 0x7f3f700b1360 2023-06-09T04:35:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.461+0000 7f3f75c53700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f700af0f0 con 0x7f3f700cd7e0 2023-06-09T04:35:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.461+0000 7f3f75c53700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f700ae270 con 0x7f3f700ae630 2023-06-09T04:35:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.461+0000 7f3f6f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f700b1360 0x7f3f700b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36534/0 (socket says 172.21.15.92:36534) 2023-06-09T04:35:31.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.461+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/3261251851 learned_addr learned my addr 172.21.15.92:0/3261251851 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6effd700 1 --2- 172.21.15.92:0/3261251851 >> v2:172.21.15.123:3300/0 conn(0x7f3f700ae630 0x7f3f700aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154635241 0 0) 0x7f3f700adde0 con 0x7f3f700b1360 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f580035b0 con 0x7f3f700b1360 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3031343237 0 0) 0x7f3f580035b0 con 0x7f3f700b1360 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 >> v2:172.21.15.123:3300/0 conn(0x7f3f700ae630 msgr2=0x7f3f700aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 --2- 172.21.15.92:0/3261251851 >> v2:172.21.15.123:3300/0 conn(0x7f3f700ae630 0x7f3f700aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 >> v1:172.21.15.123:6789/0 conn(0x7f3f700cd7e0 legacy=0x7f3f700cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700ad4b0 con 0x7f3f700b1360 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f3f600017f0 con 0x7f3f700b1360 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 >> v1:172.21.15.92:6789/0 conn(0x7f3f700b1360 legacy=0x7f3f700b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f58004850 0x7f3f58004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f58005200 0x7f3f5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f700ae270 con 0x7f3f58003a00 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5800c080 con 0x7f3f58004850 2023-06-09T04:35:31.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5800c240 con 0x7f3f58005200 2023-06-09T04:35:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6effd700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f58004850 0x7f3f58004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6ffff700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f58005200 0x7f3f5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f58005200 msgr2=0x7f3f5800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f58005200 0x7f3f5800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f58004850 msgr2=0x7f3f58004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:31.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f58004850 0x7f3f58004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/3261251851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580035b0 con 0x7f3f58003a00 2023-06-09T04:35:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6ffff700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f58005200 0x7f3f5800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.462+0000 7f3f6ffff700 1 -- 172.21.15.92:0/3261251851 reap_dead start 2023-06-09T04:35:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f58003e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c240 tx=0x7f3f58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ff9ed8e02e5383 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000cde0 con 0x7f3f58003a00 2023-06-09T04:35:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f60007b80 con 0x7f3f58003a00 2023-06-09T04:35:31.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f6e7fc700 1 -- 172.21.15.92:0/3261251851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f600072d0 con 0x7f3f58003a00 2023-06-09T04:35:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f75c53700 1 -- 172.21.15.92:0/3261251851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 msgr2=0x7f3f58003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f75c53700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f58003e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c240 tx=0x7f3f58005880 comp rx=0 tx=0).stop 2023-06-09T04:35:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f75c53700 1 -- 172.21.15.92:0/3261251851 shutdown_connections 2023-06-09T04:35:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f75c53700 1 --2- 172.21.15.92:0/3261251851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f58003e10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f75c53700 1 -- 172.21.15.92:0/3261251851 >> 172.21.15.92:0/3261251851 conn(0x7f3f70098c10 msgr2=0x7f3f700cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f75c53700 1 -- 172.21.15.92:0/3261251851 shutdown_connections 2023-06-09T04:35:31.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.463+0000 7f3f75c53700 1 -- 172.21.15.92:0/3261251851 wait complete. 2023-06-09T04:35:31.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.464+0000 7f3f75c53700 1 Processor -- start 2023-06-09T04:35:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f75c53700 1 -- start start 2023-06-09T04:35:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f75c53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f700c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f75c53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f701579f0 0x7f3f70157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f75c53700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f701f5450 0x7f3f700cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f75c53700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701f5ca0 con 0x7f3f58003a00 2023-06-09T04:35:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f75c53700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701f5f80 con 0x7f3f701579f0 2023-06-09T04:35:31.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f75c53700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f701f6260 con 0x7f3f701f5450 2023-06-09T04:35:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f700c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.465+0000 7f3f6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f700c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38214/0 (socket says 172.21.15.92:38214) 2023-06-09T04:35:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/1265213450 learned_addr learned my addr 172.21.15.92:0/1265213450 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6ffff700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f701f5450 0x7f3f700cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6effd700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f701579f0 0x7f3f70157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f701f5450 msgr2=0x7f3f700cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f701f5450 0x7f3f700cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f701579f0 msgr2=0x7f3f70157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f701579f0 0x7f3f70157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6f7fe700 1 -- 172.21.15.92:0/1265213450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700b43a0 con 0x7f3f58003a00 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6ffff700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f701f5450 0x7f3f700cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6effd700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f701579f0 0x7f3f70157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f6f7fe700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f700c6920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f60007d30 tx=0x7f3f60001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f57fff700 1 -- 172.21.15.92:0/1265213450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60005a30 con 0x7f3f58003a00 2023-06-09T04:35:31.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f57fff700 1 -- 172.21.15.92:0/1265213450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f60005b90 con 0x7f3f58003a00 2023-06-09T04:35:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f70157780 con 0x7f3f58003a00 2023-06-09T04:35:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f57fff700 1 -- 172.21.15.92:0/1265213450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60000f70 con 0x7f3f58003a00 2023-06-09T04:35:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.466+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f701f58c0 con 0x7f3f58003a00 2023-06-09T04:35:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.467+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f700b41e0 con 0x7f3f58003a00 2023-06-09T04:35:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.467+0000 7f3f57fff700 1 -- 172.21.15.92:0/1265213450 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f6001e430 con 0x7f3f58003a00 2023-06-09T04:35:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.468+0000 7f3f57fff700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06ace0 0x7f3f5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:31.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.468+0000 7f3f57fff700 1 -- 172.21.15.92:0/1265213450 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f6008a770 con 0x7f3f58003a00 2023-06-09T04:35:31.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.468+0000 7f3f6effd700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06ace0 0x7f3f5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:31.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.468+0000 7f3f6effd700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06ace0 0x7f3f5c06d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f700aee50 tx=0x7f3f64009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:31.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.472+0000 7f3f57fff700 1 -- 172.21.15.92:0/1265213450 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f600559d0 con 0x7f3f58003a00 2023-06-09T04:35:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.702+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f700740e0 con 0x7f3f58003a00 2023-06-09T04:35:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.702+0000 7f3f57fff700 1 -- 172.21.15.92:0/1265213450 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f3f60058ff0 con 0x7f3f58003a00 2023-06-09T04:35:31.705 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:31.705 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:35:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06ace0 msgr2=0x7f3f5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06ace0 0x7f3f5c06d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f700aee50 tx=0x7f3f64009210 comp rx=0 tx=0).stop 2023-06-09T04:35:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 msgr2=0x7f3f700c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f700c6920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f60007d30 tx=0x7f3f60001b80 comp rx=0 tx=0).stop 2023-06-09T04:35:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 shutdown_connections 2023-06-09T04:35:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3f5c06ace0 0x7f3f5c06d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f701f5450 0x7f3f700cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f58003a00 0x7f3f700c6920 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 --2- 172.21.15.92:0/1265213450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f701579f0 0x7f3f70157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:31.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 >> 172.21.15.92:0/1265213450 conn(0x7f3f70098c10 msgr2=0x7f3f700b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:31.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.704+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 shutdown_connections 2023-06-09T04:35:31.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:31.705+0000 7f3f75c53700 1 -- 172.21.15.92:0/1265213450 wait complete. 2023-06-09T04:35:31.708 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:35:32.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-09T04:35:17.412184+0000', 'modified': '2023-06-09T04:35:20.426961+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6835/218299678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6833', 'nonce': 218299678}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 218299678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-09T04:35:32.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-09T04:35:32.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-06-09T04:35:32.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.242+0000 7fb225ccd700 1 Processor -- start 2023-06-09T04:35:32.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.242+0000 7fb225ccd700 1 -- start start 2023-06-09T04:35:32.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb225ccd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb2200ae630 0x7fb2200aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb225ccd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2200cd7e0 0x7fb2200cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb225ccd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2200adde0 con 0x7fb2200b1360 2023-06-09T04:35:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb225ccd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2200ae270 con 0x7fb2200cd7e0 2023-06-09T04:35:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb225ccd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2200af0f0 con 0x7fb2200ae630 2023-06-09T04:35:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2200cd7e0 0x7fb2200cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2200cd7e0 0x7fb2200cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38226/0 (socket says 172.21.15.92:38226) 2023-06-09T04:35:32.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 -- 172.21.15.92:0/1615063711 learned_addr learned my addr 172.21.15.92:0/1615063711 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21effd700 1 --2- 172.21.15.92:0/1615063711 >> v2:172.21.15.123:3300/0 conn(0x7fb2200ae630 0x7fb2200aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 -- 172.21.15.92:0/1615063711 >> v2:172.21.15.123:3300/0 conn(0x7fb2200ae630 msgr2=0x7fb2200aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 --2- 172.21.15.92:0/1615063711 >> v2:172.21.15.123:3300/0 conn(0x7fb2200ae630 0x7fb2200aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 -- 172.21.15.92:0/1615063711 >> v1:172.21.15.123:6789/0 conn(0x7fb2200b1360 legacy=0x7fb2200b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 -- 172.21.15.92:0/1615063711 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2200ad4b0 con 0x7fb2200cd7e0 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21effd700 1 --2- 172.21.15.92:0/1615063711 >> v2:172.21.15.123:3300/0 conn(0x7fb2200ae630 0x7fb2200aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21ffff700 1 --2- 172.21.15.92:0/1615063711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200cd7e0 0x7fb2200cdc20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2200721a0 tx=0x7fb21800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdc111cf25f7dc6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.243+0000 7fb21e7fc700 1 -- 172.21.15.92:0/1615063711 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21800e070 con 0x7fb2200cd7e0 2023-06-09T04:35:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb21e7fc700 1 -- 172.21.15.92:0/1615063711 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218003a20 con 0x7fb2200cd7e0 2023-06-09T04:35:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb21e7fc700 1 -- 172.21.15.92:0/1615063711 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218003dd0 con 0x7fb2200cd7e0 2023-06-09T04:35:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 -- 172.21.15.92:0/1615063711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200cd7e0 msgr2=0x7fb2200cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 --2- 172.21.15.92:0/1615063711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200cd7e0 0x7fb2200cdc20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2200721a0 tx=0x7fb21800b330 comp rx=0 tx=0).stop 2023-06-09T04:35:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 -- 172.21.15.92:0/1615063711 shutdown_connections 2023-06-09T04:35:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 --2- 172.21.15.92:0/1615063711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200cd7e0 0x7fb2200cdc20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 --2- 172.21.15.92:0/1615063711 >> v2:172.21.15.123:3300/0 conn(0x7fb2200ae630 0x7fb2200aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 -- 172.21.15.92:0/1615063711 >> 172.21.15.92:0/1615063711 conn(0x7fb220098c10 msgr2=0x7fb2200cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 -- 172.21.15.92:0/1615063711 shutdown_connections 2023-06-09T04:35:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.244+0000 7fb225ccd700 1 -- 172.21.15.92:0/1615063711 wait complete. 2023-06-09T04:35:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.245+0000 7fb225ccd700 1 Processor -- start 2023-06-09T04:35:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.245+0000 7fb225ccd700 1 -- start start 2023-06-09T04:35:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb225ccd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb2200ae630 0x7fb22014f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb225ccd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200b1360 0x7fb2200c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb225ccd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2200cd7e0 0x7fb22006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb225ccd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200c5680 con 0x7fb2200b1360 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb225ccd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2200c51f0 con 0x7fb2200cd7e0 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb225ccd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22014fc00 con 0x7fb2200ae630 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200b1360 0x7fb2200c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200b1360 0x7fb2200c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38230/0 (socket says 172.21.15.92:38230) 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb21effd700 1 -- 172.21.15.92:0/915738092 learned_addr learned my addr 172.21.15.92:0/915738092 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb21effd700 1 -- 172.21.15.92:0/915738092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb2200ae630 msgr2=0x7fb22014f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb21effd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb2200ae630 0x7fb22014f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb21effd700 1 -- 172.21.15.92:0/915738092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2200cd7e0 msgr2=0x7fb22006cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.246+0000 7fb21effd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2200cd7e0 0x7fb22006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.247+0000 7fb21effd700 1 -- 172.21.15.92:0/915738092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218009b30 con 0x7fb2200b1360 2023-06-09T04:35:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.247+0000 7fb21effd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200b1360 0x7fb2200c3d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb21000c750 tx=0x7fb21000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.247+0000 7fb207fff700 1 -- 172.21.15.92:0/915738092 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210011800 con 0x7fb2200b1360 2023-06-09T04:35:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.247+0000 7fb207fff700 1 -- 172.21.15.92:0/915738092 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb210011960 con 0x7fb2200b1360 2023-06-09T04:35:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.247+0000 7fb207fff700 1 -- 172.21.15.92:0/915738092 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210011cd0 con 0x7fb2200b1360 2023-06-09T04:35:32.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.247+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2200c45e0 con 0x7fb2200b1360 2023-06-09T04:35:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.247+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2200c4ba0 con 0x7fb2200b1360 2023-06-09T04:35:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.248+0000 7fb207fff700 1 -- 172.21.15.92:0/915738092 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb210021350 con 0x7fb2200b1360 2023-06-09T04:35:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.249+0000 7fb207fff700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb20c06ad10 0x7fb20c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.249+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb20c06ad10 0x7fb20c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.249+0000 7fb207fff700 1 -- 172.21.15.92:0/915738092 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb2100893b0 con 0x7fb2200b1360 2023-06-09T04:35:32.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.249+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb20c06ad10 0x7fb20c06d1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb214003e80 tx=0x7fb214007a90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.251+0000 7fb231e52b80 1 Processor -- start 2023-06-09T04:35:32.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.251+0000 7fb231e52b80 1 -- start start 2023-06-09T04:35:32.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da4f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da53b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da9d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb231e52b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2d9593b0 con 0x555a2da536c0 2023-06-09T04:35:32.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2d95ac60 con 0x555a2da9cc70 2023-06-09T04:35:32.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a2d9ad570 con 0x555a2da532f0 2023-06-09T04:35:32.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da53b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da53b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38238/0 (socket says 172.21.15.92:38238) 2023-06-09T04:35:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 -- 172.21.15.92:0/2548670401 learned_addr learned my addr 172.21.15.92:0/2548670401 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da4f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21ffff700 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da9d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 -- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 msgr2=0x555a2da4f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da4f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 -- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 msgr2=0x555a2da9d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da9d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 -- 172.21.15.92:0/2548670401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a2d7d4d90 con 0x555a2da536c0 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21ffff700 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da9d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb21effd700 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da53b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb210058050 tx=0x7fb21005c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23402b6f86c74430 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb2057fa700 1 -- 172.21.15.92:0/2548670401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210057600 con 0x555a2da536c0 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb2057fa700 1 -- 172.21.15.92:0/2548670401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb210057760 con 0x555a2da536c0 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.252+0000 7fb2057fa700 1 -- 172.21.15.92:0/2548670401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100578d0 con 0x555a2da536c0 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 -- 172.21.15.92:0/2548670401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 msgr2=0x555a2da53b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da53b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb210058050 tx=0x7fb21005c250 comp rx=0 tx=0).stop 2023-06-09T04:35:32.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 -- 172.21.15.92:0/2548670401 shutdown_connections 2023-06-09T04:35:32.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da9d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da4f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2548670401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da53b30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 -- 172.21.15.92:0/2548670401 >> 172.21.15.92:0/2548670401 conn(0x555a2da51ce0 msgr2=0x555a2da9a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:32.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 -- 172.21.15.92:0/2548670401 shutdown_connections 2023-06-09T04:35:32.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.253+0000 7fb231e52b80 1 -- 172.21.15.92:0/2548670401 wait complete. 2023-06-09T04:35:32.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.254+0000 7fb231e52b80 1 -- start start 2023-06-09T04:35:32.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.254+0000 7fb231e52b80 1 Processor -- start 2023-06-09T04:35:32.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2daa9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da97f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb231e52b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2d9a6a00 con 0x555a2da536c0 2023-06-09T04:35:32.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2d946a70 con 0x555a2da9cc70 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a2d946770 con 0x555a2da532f0 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb231e52b80 10 client.0 _ll_register_callbacks cb 0x555a2d944630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da97f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da97f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38242/0 (socket says 172.21.15.92:38242) 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 -- 172.21.15.92:0/2764376590 learned_addr learned my addr 172.21.15.92:0/2764376590 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2daa9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 -- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 msgr2=0x555a2daa9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2daa9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 -- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 msgr2=0x555a2da984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21ffff700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 -- 172.21.15.92:0/2764376590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2200c4ba0 con 0x555a2da536c0 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2daa9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb21effd700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da97f90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb210056460 tx=0x7fb210000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97aa9a682458901 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.255+0000 7fb1fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-06-09T04:35:32.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/2764376590 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100874e0 con 0x555a2da536c0 2023-06-09T04:35:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/2764376590 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb210087b00 con 0x555a2da536c0 2023-06-09T04:35:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a2da47850 con 0x555a2da536c0 2023-06-09T04:35:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/2764376590 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb210054450 con 0x555a2da536c0 2023-06-09T04:35:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 10 client.4662 fetch_fsmap learned FSMap version 8 2023-06-09T04:35:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x555a2d7bdc10 con 0x555a2da536c0 2023-06-09T04:35:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/2764376590 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a2da47850 con 0x555a2da536c0 2023-06-09T04:35:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/2764376590 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb210087cc0 con 0x555a2da536c0 2023-06-09T04:35:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-06-09T04:35:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2023-06-09T04:35:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 20 client.4662 populate_metadata read hostname 'smithi092' 2023-06-09T04:35:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da7d490 0x555a2da7d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 4 client.4662 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-09T04:35:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 --> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a2d99f9e0 con 0x555a2da7d490 2023-06-09T04:35:32.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.256+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da7d490 0x555a2da7d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.257+0000 7fb1fe7fc700 10 client.4662 ms_handle_connect on v2:172.21.15.123:6833/218299678 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.257+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da7d490 0x555a2da7d860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e963cd69890b0b6 server_cookie=ff0f274f7b82e33e in_seq=0 out_seq=0 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.257+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/2764376590 <== mds.0 v2:172.21.15.123:6833/218299678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a2d99f9e0 con 0x555a2da7d490 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.257+0000 7fb1fe7fc700 10 client.4662 handle_command_reply: tid=0 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.257+0000 7fb231e52b80 1 client.4662 shutdown 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 msgr2=0x555a2da97f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da97f90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb210056460 tx=0x7fb210000bc0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 shutdown_connections 2023-06-09T04:35:32.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da7d490 0x555a2da7d860 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2daa9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb1fe7fc700 0 client.4662 ms_handle_reset on v2:172.21.15.123:6833/218299678 2023-06-09T04:35:32.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 --2- 172.21.15.92:0/2764376590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2da97f90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 >> 172.21.15.92:0/2764376590 conn(0x555a2da51ce0 msgr2=0x555a2da4f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:32.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 shutdown_connections 2023-06-09T04:35:32.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 1 -- 172.21.15.92:0/2764376590 wait complete. 2023-06-09T04:35:32.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.258+0000 7fb231e52b80 20 client.4662 trim_cache size 0 max 16384 2023-06-09T04:35:32.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.275+0000 7fb231e52b80 1 Processor -- start 2023-06-09T04:35:32.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.275+0000 7fb231e52b80 1 -- start start 2023-06-09T04:35:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da532f0 0x555a2da98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da536c0 0x555a2da9e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb231e52b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2d82a8d0 con 0x555a2da532f0 2023-06-09T04:35:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2da9b2d0 con 0x555a2da9cc70 2023-06-09T04:35:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a2da7cfc0 con 0x555a2da536c0 2023-06-09T04:35:32.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da532f0 0x555a2da98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da532f0 0x555a2da98130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38250/0 (socket says 172.21.15.92:38250) 2023-06-09T04:35:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 -- 172.21.15.92:0/809349572 learned_addr learned my addr 172.21.15.92:0/809349572 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 -- 172.21.15.92:0/809349572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da536c0 msgr2=0x555a2da9e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/809349572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da536c0 0x555a2da9e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 -- 172.21.15.92:0/809349572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 msgr2=0x555a2da49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/809349572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 -- 172.21.15.92:0/809349572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a2da77dc0 con 0x555a2da532f0 2023-06-09T04:35:32.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.276+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/809349572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da532f0 0x555a2da98130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb2140045d0 tx=0x7fb214012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df380b1533e2aa2c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb205ffb700 1 -- 172.21.15.92:0/809349572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214023430 con 0x555a2da532f0 2023-06-09T04:35:32.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb205ffb700 1 -- 172.21.15.92:0/809349572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb214023590 con 0x555a2da532f0 2023-06-09T04:35:32.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb205ffb700 1 -- 172.21.15.92:0/809349572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214023c00 con 0x555a2da532f0 2023-06-09T04:35:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 -- 172.21.15.92:0/809349572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da532f0 msgr2=0x555a2da98130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 --2- 172.21.15.92:0/809349572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da532f0 0x555a2da98130 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb2140045d0 tx=0x7fb214012000 comp rx=0 tx=0).stop 2023-06-09T04:35:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 -- 172.21.15.92:0/809349572 shutdown_connections 2023-06-09T04:35:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 --2- 172.21.15.92:0/809349572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2da49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 --2- 172.21.15.92:0/809349572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da532f0 0x555a2da98130 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 --2- 172.21.15.92:0/809349572 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da536c0 0x555a2da9e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 -- 172.21.15.92:0/809349572 >> 172.21.15.92:0/809349572 conn(0x555a2da51ce0 msgr2=0x555a2da82e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 -- 172.21.15.92:0/809349572 shutdown_connections 2023-06-09T04:35:32.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.277+0000 7fb231e52b80 1 -- 172.21.15.92:0/809349572 wait complete. 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.278+0000 7fb231e52b80 1 -- start start 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.278+0000 7fb231e52b80 1 Processor -- start 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da7e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2db03530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2db03a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2dab2fe0 con 0x555a2da536c0 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2da49880 con 0x555a2da9cc70 2023-06-09T04:35:32.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a2da7e660 con 0x555a2da532f0 2023-06-09T04:35:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 10 client.0 _ll_register_callbacks cb 0x555a2d944630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T04:35:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2db03530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2db03530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38258/0 (socket says 172.21.15.92:38258) 2023-06-09T04:35:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 -- 172.21.15.92:0/1686617370 learned_addr learned my addr 172.21.15.92:0/1686617370 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da7e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21ffff700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2db03a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 -- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 msgr2=0x555a2da7e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da7e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 -- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 msgr2=0x555a2db03a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2db03a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 -- 172.21.15.92:0/1686617370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb214014500 con 0x555a2da536c0 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21ffff700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2db03a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb21effd700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2db03530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb210057be0 tx=0x7fb210087640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7d92de6a74aca4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb1fe7fc700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-06-09T04:35:32.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/1686617370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100874e0 con 0x555a2da536c0 2023-06-09T04:35:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/1686617370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb210054220 con 0x555a2da536c0 2023-06-09T04:35:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb210057760 con 0x555a2da536c0 2023-06-09T04:35:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/1686617370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb210064070 con 0x555a2da536c0 2023-06-09T04:35:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 10 client.4668 fetch_fsmap learned FSMap version 8 2023-06-09T04:35:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x555a2d7fb200 con 0x555a2da536c0 2023-06-09T04:35:32.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/1686617370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb210057760 con 0x555a2da536c0 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.279+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/1686617370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb210067070 con 0x555a2da536c0 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 20 client.4668 populate_metadata read hostname 'smithi092' 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da8fe40 0x555a2da90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 4 client.4668 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 --> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a2d962000 con 0x555a2da8fe40 2023-06-09T04:35:32.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da8fe40 0x555a2da90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb1fe7fc700 10 client.4668 ms_handle_connect on v2:172.21.15.123:6833/218299678 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb21f7fe700 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da8fe40 0x555a2da90210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36bc744941fbf531 server_cookie=a0c1d0a025cae056 in_seq=0 out_seq=0 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb1fe7fc700 1 -- 172.21.15.92:0/1686617370 <== mds.0 v2:172.21.15.123:6833/218299678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x555a2d962000 con 0x555a2da8fe40 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb1fe7fc700 10 client.4668 handle_command_reply: tid=0 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 client.4668 shutdown 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 msgr2=0x555a2db03530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2db03530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb210057be0 tx=0x7fb210087640 comp rx=0 tx=0).stop 2023-06-09T04:35:32.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 shutdown_connections 2023-06-09T04:35:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:6833/218299678,v1:172.21.15.123:6835/218299678] conn(0x555a2da8fe40 0x555a2da90210 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555a2da9cc70 0x555a2db03a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555a2da532f0 0x555a2da7e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 --2- 172.21.15.92:0/1686617370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a2da536c0 0x555a2db03530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb1fe7fc700 0 client.4668 ms_handle_reset on v2:172.21.15.123:6833/218299678 2023-06-09T04:35:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.280+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 >> 172.21.15.92:0/1686617370 conn(0x555a2da51ce0 msgr2=0x555a2da9b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:32.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 shutdown_connections 2023-06-09T04:35:32.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb231e52b80 1 -- 172.21.15.92:0/1686617370 wait complete. 2023-06-09T04:35:32.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb231e52b80 20 client.4668 trim_cache size 0 max 16384 2023-06-09T04:35:32.292 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-06-09T04:35:32.292 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "064bde79-2943-47e5-b941-05f234bd883f", 2023-06-09T04:35:32.292 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4296, 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 14.867757299000001, 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 8, 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 24, 2023-06-09T04:35:32.293 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 24, 2023-06-09T04:35:32.294 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 20.170145640000001 2023-06-09T04:35:32.294 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-06-09T04:35:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb20c06ad10 msgr2=0x7fb20c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb20c06ad10 0x7fb20c06d1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb214003e80 tx=0x7fb214007a90 comp rx=0 tx=0).stop 2023-06-09T04:35:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200b1360 msgr2=0x7fb2200c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200b1360 0x7fb2200c3d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb21000c750 tx=0x7fb21000cb10 comp rx=0 tx=0).stop 2023-06-09T04:35:32.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 shutdown_connections 2023-06-09T04:35:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb20c06ad10 0x7fb20c06d1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2200cd7e0 0x7fb22006cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb2200ae630 0x7fb22014f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 --2- 172.21.15.92:0/915738092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2200b1360 0x7fb2200c3d60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 >> 172.21.15.92:0/915738092 conn(0x7fb220098c10 msgr2=0x7fb2200b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 shutdown_connections 2023-06-09T04:35:32.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.281+0000 7fb225ccd700 1 -- 172.21.15.92:0/915738092 wait complete. 2023-06-09T04:35:32.297 INFO:teuthology.run_tasks:Running task kclient... 2023-06-09T04:35:32.320 INFO:tasks.kclient:Mounting kernel clients... 2023-06-09T04:35:32.321 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-06-09T04:35:32.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.322 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-06-09T04:35:32.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-06-09T04:35:32.406 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-06-09T04:35:32.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:35:32.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.409 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-06-09T04:35:32.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-06-09T04:35:32.533 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-06-09T04:35:32.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:35:32.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.536 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-06-09T04:35:32.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-06-09T04:35:32.633 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-06-09T04:35:32.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:35:32.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.635 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-06-09T04:35:32.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:32.696 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-06-09T04:35:32.768 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-06-09T04:35:32.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:35:32.770 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-06-09T04:35:32.770 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-09T04:35:32.770 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-09T04:35:32.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-09T04:35:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.890+0000 7f9701026700 1 Processor -- start 2023-06-09T04:35:32.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.890+0000 7f9701026700 1 -- start start 2023-06-09T04:35:32.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f9701026700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a500 0x7f96fc0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f9701026700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96fc0cbf50 0x7f96fc0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f9701026700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96fc0ac400 con 0x7f96fc15a8d0 2023-06-09T04:35:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f9701026700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96fc0cd7e0 con 0x7f96fc0cbf50 2023-06-09T04:35:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f9701026700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0cdda0 con 0x7f96fc15a500 2023-06-09T04:35:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96fc0cbf50 0x7f96fc0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96fc0cbf50 0x7f96fc0ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38268/0 (socket says 172.21.15.92:38268) 2023-06-09T04:35:32.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fa59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96fc15a8d0 0x7f96fc069db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36598/0 (socket says 172.21.15.92:36598) 2023-06-09T04:35:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 -- 172.21.15.92:0/3601179380 learned_addr learned my addr 172.21.15.92:0/3601179380 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fad9d700 1 --2- 172.21.15.92:0/3601179380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a500 0x7f96fc0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 -- 172.21.15.92:0/3601179380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a500 msgr2=0x7f96fc0cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 --2- 172.21.15.92:0/3601179380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a500 0x7f96fc0cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 -- 172.21.15.92:0/3601179380 >> v1:172.21.15.92:6789/0 conn(0x7f96fc15a8d0 legacy=0x7f96fc069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96f9d9b700 1 -- 172.21.15.92:0/3601179380 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624336666 0 0) 0x7f96fc0ac400 con 0x7f96fc15a8d0 2023-06-09T04:35:32.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 -- 172.21.15.92:0/3601179380 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc0aeca0 con 0x7f96fc0cbf50 2023-06-09T04:35:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96fb59e700 1 --2- 172.21.15.92:0/3601179380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc0ce230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009e50 tx=0x7f96f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead156e5fffe937c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96f9d9b700 1 -- 172.21.15.92:0/3601179380 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f400e070 con 0x7f96fc0cbf50 2023-06-09T04:35:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96f9d9b700 1 -- 172.21.15.92:0/3601179380 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f400b9c0 con 0x7f96fc0cbf50 2023-06-09T04:35:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.891+0000 7f96f9d9b700 1 -- 172.21.15.92:0/3601179380 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f400bd70 con 0x7f96fc0cbf50 2023-06-09T04:35:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 -- 172.21.15.92:0/3601179380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 msgr2=0x7f96fc0ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 --2- 172.21.15.92:0/3601179380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc0ce230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009e50 tx=0x7f96f400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:32.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 -- 172.21.15.92:0/3601179380 shutdown_connections 2023-06-09T04:35:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 --2- 172.21.15.92:0/3601179380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a500 0x7f96fc0cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 --2- 172.21.15.92:0/3601179380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc0ce230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 -- 172.21.15.92:0/3601179380 >> 172.21.15.92:0/3601179380 conn(0x7f96fc098c10 msgr2=0x7f96fc0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 -- 172.21.15.92:0/3601179380 shutdown_connections 2023-06-09T04:35:32.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.892+0000 7f9701026700 1 -- 172.21.15.92:0/3601179380 wait complete. 2023-06-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.893+0000 7f9701026700 1 Processor -- start 2023-06-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.893+0000 7f9701026700 1 -- start start 2023-06-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f9701026700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc1f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f9701026700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96fc15a500 0x7f96fc0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f9701026700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a8d0 0x7f96fc0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f9701026700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc14c820 con 0x7f96fc0cbf50 2023-06-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f9701026700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1f3580 con 0x7f96fc15a500 2023-06-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f9701026700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc1f3820 con 0x7f96fc15a8d0 2023-06-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f96fad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc1f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f96fad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc1f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38270/0 (socket says 172.21.15.92:38270) 2023-06-09T04:35:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f96fad9d700 1 -- 172.21.15.92:0/3044643787 learned_addr learned my addr 172.21.15.92:0/3044643787 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f96fb59e700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a8d0 0x7f96fc0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.894+0000 7f96fa59c700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96fc15a500 0x7f96fc0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96fad9d700 1 -- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a8d0 msgr2=0x7f96fc0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96fad9d700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a8d0 0x7f96fc0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96fad9d700 1 -- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96fc15a500 msgr2=0x7f96fc0b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96fad9d700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96fc15a500 0x7f96fc0b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:32.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96fad9d700 1 -- 172.21.15.92:0/3044643787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f4009b30 con 0x7f96fc0cbf50 2023-06-09T04:35:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96fad9d700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc1f3040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e730 tx=0x7f96f000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96e37fe700 1 -- 172.21.15.92:0/3044643787 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000c070 con 0x7f96fc0cbf50 2023-06-09T04:35:32.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96e37fe700 1 -- 172.21.15.92:0/3044643787 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0013610 con 0x7f96fc0cbf50 2023-06-09T04:35:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96fc1f3b00 con 0x7f96fc0cbf50 2023-06-09T04:35:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f96e37fe700 1 -- 172.21.15.92:0/3044643787 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f00139c0 con 0x7f96fc0cbf50 2023-06-09T04:35:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.895+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc14ca30 con 0x7f96fc0cbf50 2023-06-09T04:35:32.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.896+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96fc14cbc0 con 0x7f96fc0cbf50 2023-06-09T04:35:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.896+0000 7f96e37fe700 1 -- 172.21.15.92:0/3044643787 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96f0013b20 con 0x7f96fc0cbf50 2023-06-09T04:35:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.897+0000 7f96e37fe700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96e806ace0 0x7f96e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:32.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.897+0000 7f96e37fe700 1 -- 172.21.15.92:0/3044643787 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96f0088d90 con 0x7f96fc0cbf50 2023-06-09T04:35:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.897+0000 7f96fa59c700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96e806ace0 0x7f96e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:32.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.897+0000 7f96fa59c700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96e806ace0 0x7f96e806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96fc14c4e0 tx=0x7f96ec007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:32.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:32.901+0000 7f96e37fe700 1 -- 172.21.15.92:0/3044643787 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f96f0053ed0 con 0x7f96fc0cbf50 2023-06-09T04:35:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.135+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96fc0740e0 con 0x7f96fc0cbf50 2023-06-09T04:35:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.135+0000 7f96e37fe700 1 -- 172.21.15.92:0/3044643787 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f96f00574f0 con 0x7f96fc0cbf50 2023-06-09T04:35:33.137 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-09T04:35:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.137+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96e806ace0 msgr2=0x7f96e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.137+0000 7f9701026700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96e806ace0 0x7f96e806d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96fc14c4e0 tx=0x7f96ec007e70 comp rx=0 tx=0).stop 2023-06-09T04:35:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.137+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 msgr2=0x7f96fc1f3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:33.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.137+0000 7f9701026700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc1f3040 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e730 tx=0x7f96f000ea40 comp rx=0 tx=0).stop 2023-06-09T04:35:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 shutdown_connections 2023-06-09T04:35:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96e806ace0 0x7f96e806d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96fc15a8d0 0x7f96fc0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96fc0cbf50 0x7f96fc1f3040 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 --2- 172.21.15.92:0/3044643787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96fc15a500 0x7f96fc0b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 >> 172.21.15.92:0/3044643787 conn(0x7f96fc098c10 msgr2=0x7f96fc1502f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 shutdown_connections 2023-06-09T04:35:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:33.138+0000 7f9701026700 1 -- 172.21.15.92:0/3044643787 wait complete. 2023-06-09T04:35:33.151 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-09T04:35:33.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-09T04:35:33.152 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-06-09T04:35:33.152 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-09T04:35:33.153 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-09T04:35:33.153 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-09T04:35:33.153 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-09T04:35:33.153 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-09T04:35:33.154 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-09T04:35:33.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.155 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-06-09T04:35:33.172 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:33.172 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:33.173 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:33.173 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:33.174 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-09T04:35:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-09T04:35:33.174 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42432sec preferred_lft 42432sec 2023-06-09T04:35:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::e3d4:5405:af43:207f/64 scope link noprefixroute 2023-06-09T04:35:33.175 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:33.175 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-09T04:35:33.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.176 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:33.176 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:33.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-06-09T04:35:33.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-06-09T04:35:33.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-06-09T04:35:33.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-09T04:35:33.178 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:33.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.344 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-09T04:35:33.371 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-06-09T04:35:33.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.373 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-06-09T04:35:33.441 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-06-09T04:35:33.441 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-09T04:35:33.442 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-09T04:35:33.442 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-09T04:35:33.442 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-09T04:35:33.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.443 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:33.443 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:33.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-09T04:35:33.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-09T04:35:33.444 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-09T04:35:33.444 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:33.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.685 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-06-09T04:35:33.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.702 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-06-09T04:35:33.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.759 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:33.759 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:33.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-09T04:35:33.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-06-09T04:35:33.760 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:33.852 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-06-09T04:35:33.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:33.853 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:33.853 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:33.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-06-09T04:35:33.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-09T04:35:33.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-06-09T04:35:33.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-06-09T04:35:33.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-06-09T04:35:33.854 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:34.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:34.038 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:34.038 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:34.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-06-09T04:35:34.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-06-09T04:35:34.039 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:34.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:34.141 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-09T04:35:34.160 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-09T04:35:34.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:34.161 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-09T04:35:34.216 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-06-09T04:35:34.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:34.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:34.310 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:34.310 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:34.417 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:34.417 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:34.417 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:34.417 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:34.417 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:34.418 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:34.418 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:34.418 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:34.497 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:34.498 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:34.498 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:34.498 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:34.498 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:34.498 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:34.498 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:34.499 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:34.499 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-06-09T04:35:34.499 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-06-09T04:35:34.499 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=064bde79-2943-47e5-b941-05f234bd883f" will pass to kernel 2023-06-09T04:35:34.500 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-09T04:35:34.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:34.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-09T04:35:34.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:34.528 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:34.614 INFO:teuthology.orchestra.run.smithi092.stdout:client4511 2023-06-09T04:35:34.615 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:34.615 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:34.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:34.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:34.718 INFO:teuthology.orchestra.run.smithi092.stdout:client4511 2023-06-09T04:35:34.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:34.719 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:34.767 INFO:teuthology.orchestra.run.smithi092.stdout:client4511 2023-06-09T04:35:34.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:34.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/064bde79-2943-47e5-b941-05f234bd883f.client4511/status') 2023-06-09T04:35:34.820 INFO:teuthology.orchestra.run.smithi092.stdout:instance: client.4511 (3)172.21.15.92:0/1619409395 2023-06-09T04:35:34.821 INFO:teuthology.orchestra.run.smithi092.stdout:blocklisted: false 2023-06-09T04:35:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2023-06-09T04:35:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2023-06-09T04:35:34.821 INFO:teuthology.orchestra.run.smithi092.stderr:70 bytes copied, 2.3736e-05 s, 2.9 MB/s 2023-06-09T04:35:34.822 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-06-09T04:35:34.823 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-09T04:35:34.823 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-09T04:35:34.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-09T04:35:34.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.941+0000 7fa544b2f700 1 Processor -- start 2023-06-09T04:35:34.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.941+0000 7fa544b2f700 1 -- start start 2023-06-09T04:35:34.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.942+0000 7fa544b2f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400b1360 0x7fa5400b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:34.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.942+0000 7fa544b2f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa5400ae630 0x7fa5400aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:34.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.942+0000 7fa544b2f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5400adde0 con 0x7fa5400ac400 2023-06-09T04:35:34.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.942+0000 7fa544b2f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5400ae270 con 0x7fa5400ae630 2023-06-09T04:35:34.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.942+0000 7fa544b2f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400af0f0 con 0x7fa5400b1360 2023-06-09T04:35:34.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53dd9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa5400ae630 0x7fa5400aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:34.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400b1360 0x7fa5400b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:34.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400b1360 0x7fa5400b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:47192/0 (socket says 172.21.15.92:47192) 2023-06-09T04:35:34.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53ed9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa5400ac400 0x7fa5400ac880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59114/0 (socket says 172.21.15.92:59114) 2023-06-09T04:35:34.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53dd9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa5400ae630 0x7fa5400aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58692/0 (socket says 172.21.15.92:58692) 2023-06-09T04:35:34.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 -- 172.21.15.92:0/3216251623 learned_addr learned my addr 172.21.15.92:0/3216251623 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:34.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 -- 172.21.15.92:0/3216251623 >> v2:172.21.15.123:3300/0 conn(0x7fa5400ae630 msgr2=0x7fa5400aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:34.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 --2- 172.21.15.92:0/3216251623 >> v2:172.21.15.123:3300/0 conn(0x7fa5400ae630 0x7fa5400aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:34.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 -- 172.21.15.92:0/3216251623 >> v1:172.21.15.123:6789/0 conn(0x7fa5400ac400 legacy=0x7fa5400ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 -- 172.21.15.92:0/3216251623 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400b1c70 con 0x7fa5400b1360 2023-06-09T04:35:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.943+0000 7fa53e59c700 1 --2- 172.21.15.92:0/3216251623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400b1360 0x7fa5400b1730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa530009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71faa55ab512d8d1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa53d59a700 1 -- 172.21.15.92:0/3216251623 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530005e30 con 0x7fa5400b1360 2023-06-09T04:35:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa53d59a700 1 -- 172.21.15.92:0/3216251623 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa530004030 con 0x7fa5400b1360 2023-06-09T04:35:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa53d59a700 1 -- 172.21.15.92:0/3216251623 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300043e0 con 0x7fa5400b1360 2023-06-09T04:35:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 -- 172.21.15.92:0/3216251623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400b1360 msgr2=0x7fa5400b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:34.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 --2- 172.21.15.92:0/3216251623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400b1360 0x7fa5400b1730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa530006fd0 tx=0x7fa530009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 -- 172.21.15.92:0/3216251623 shutdown_connections 2023-06-09T04:35:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 --2- 172.21.15.92:0/3216251623 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400b1360 0x7fa5400b1730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 --2- 172.21.15.92:0/3216251623 >> v2:172.21.15.123:3300/0 conn(0x7fa5400ae630 0x7fa5400aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 -- 172.21.15.92:0/3216251623 >> 172.21.15.92:0/3216251623 conn(0x7fa540098c10 msgr2=0x7fa5400cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 -- 172.21.15.92:0/3216251623 shutdown_connections 2023-06-09T04:35:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.944+0000 7fa544b2f700 1 -- 172.21.15.92:0/3216251623 wait complete. 2023-06-09T04:35:34.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.946+0000 7fa544b2f700 1 Processor -- start 2023-06-09T04:35:34.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.946+0000 7fa544b2f700 1 -- start start 2023-06-09T04:35:34.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa544b2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5400ac400 0x7fa5400c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa544b2f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400ae630 0x7fa5400c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa544b2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5400b1360 0x7fa540147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa544b2f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54009d4e0 con 0x7fa5400ac400 2023-06-09T04:35:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa544b2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5400c9e80 con 0x7fa5400b1360 2023-06-09T04:35:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa544b2f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa540148bc0 con 0x7fa5400ae630 2023-06-09T04:35:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5400ac400 0x7fa5400c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:34.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5400ac400 0x7fa5400c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38288/0 (socket says 172.21.15.92:38288) 2023-06-09T04:35:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 -- 172.21.15.92:0/2794220515 learned_addr learned my addr 172.21.15.92:0/2794220515 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53dd9b700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400ae630 0x7fa5400c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53ed9d700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5400b1360 0x7fa540147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 -- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400ae630 msgr2=0x7fa5400c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400ae630 0x7fa5400c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 -- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5400b1360 msgr2=0x7fa540147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:34.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5400b1360 0x7fa540147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.947+0000 7fa53e59c700 1 -- 172.21.15.92:0/2794220515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530009500 con 0x7fa5400ac400 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.948+0000 7fa53e59c700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5400ac400 0x7fa5400c8b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa530004000 tx=0x7fa530004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.948+0000 7fa536ffd700 1 -- 172.21.15.92:0/2794220515 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa530003d90 con 0x7fa5400ac400 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.948+0000 7fa536ffd700 1 -- 172.21.15.92:0/2794220515 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53001c420 con 0x7fa5400ac400 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.948+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa540146380 con 0x7fa5400ac400 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.948+0000 7fa536ffd700 1 -- 172.21.15.92:0/2794220515 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53001c7d0 con 0x7fa5400ac400 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.948+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa540145d30 con 0x7fa5400ac400 2023-06-09T04:35:34.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.949+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa540145b90 con 0x7fa5400ac400 2023-06-09T04:35:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.949+0000 7fa536ffd700 1 -- 172.21.15.92:0/2794220515 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa53001c930 con 0x7fa5400ac400 2023-06-09T04:35:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.949+0000 7fa536ffd700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa52006ad10 0x7fa52006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.950+0000 7fa53dd9b700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa52006ad10 0x7fa52006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.950+0000 7fa536ffd700 1 -- 172.21.15.92:0/2794220515 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa53008a570 con 0x7fa5400ac400 2023-06-09T04:35:34.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.950+0000 7fa53dd9b700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa52006ad10 0x7fa52006d1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa5400cef30 tx=0x7fa528008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:34.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:34.954+0000 7fa536ffd700 1 -- 172.21.15.92:0/2794220515 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa540145b90 con 0x7fa5400ac400 2023-06-09T04:35:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.189+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa540145b90 con 0x7fa5400ac400 2023-06-09T04:35:35.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.189+0000 7fa536ffd700 1 -- 172.21.15.92:0/2794220515 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa540145b90 con 0x7fa5400ac400 2023-06-09T04:35:35.191 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-09T04:35:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa52006ad10 msgr2=0x7fa52006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa52006ad10 0x7fa52006d1c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa5400cef30 tx=0x7fa528008040 comp rx=0 tx=0).stop 2023-06-09T04:35:35.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5400ac400 msgr2=0x7fa5400c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5400ac400 0x7fa5400c8b30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa530004000 tx=0x7fa530004750 comp rx=0 tx=0).stop 2023-06-09T04:35:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 shutdown_connections 2023-06-09T04:35:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa52006ad10 0x7fa52006d1c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5400b1360 0x7fa540147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5400ac400 0x7fa5400c8b30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 --2- 172.21.15.92:0/2794220515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5400ae630 0x7fa5400c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:35.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 >> 172.21.15.92:0/2794220515 conn(0x7fa540098c10 msgr2=0x7fa5400b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 shutdown_connections 2023-06-09T04:35:35.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:35.192+0000 7fa544b2f700 1 -- 172.21.15.92:0/2794220515 wait complete. 2023-06-09T04:35:35.205 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-09T04:35:35.205 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-09T04:35:35.205 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-06-09T04:35:35.205 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-06-09T04:35:35.206 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-09T04:35:35.206 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-09T04:35:35.206 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-09T04:35:35.206 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-09T04:35:35.206 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-06-09T04:35:35.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.207 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-06-09T04:35:35.224 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.224 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.225 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:35.225 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:35.225 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:35.225 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:35.226 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-09T04:35:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-09T04:35:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42430sec preferred_lft 42430sec 2023-06-09T04:35:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::e3d4:5405:af43:207f/64 scope link noprefixroute 2023-06-09T04:35:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.227 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-09T04:35:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 2e:c9:69:7c:66:b6 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-09T04:35:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::825:61ff:fe8f:9919/64 scope link 2023-06-09T04:35:35.227 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.228 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-09T04:35:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 2e:c9:69:7c:66:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-09T04:35:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::2cc9:69ff:fe7c:66b6/64 scope link tentative 2023-06-09T04:35:35.228 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.229 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-09T04:35:35.288 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-06-09T04:35:35.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.290 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-06-09T04:35:35.348 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-06-09T04:35:35.349 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-09T04:35:35.349 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-09T04:35:35.349 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-09T04:35:35.349 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-09T04:35:35.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.350 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:35.350 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:35.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-09T04:35:35.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-09T04:35:35.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-09T04:35:35.351 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:35.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.453 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-06-09T04:35:35.469 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-09T04:35:35.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.470 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-06-09T04:35:35.526 INFO:teuthology.orchestra.run.smithi092.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-06-09T04:35:35.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.527 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:35.527 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:35.527 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-09T04:35:35.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-06-09T04:35:35.528 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:35.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-09T04:35:35.655 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.655 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.656 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-09T04:35:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:71:cf:fe:11:b8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-09T04:35:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-09T04:35:35.656 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::7871:cfff:fefe:11b8/64 scope link tentative 2023-06-09T04:35:35.657 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-09T04:35:35.746 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:35.746 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 7a:71:cf:fe:11:b8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.747 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::7871:cfff:fefe:11b8/64 scope link tentative 2023-06-09T04:35:35.748 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:35.758 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-06-09T04:35:35.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:35.759 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:35.759 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:35.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-06-09T04:35:35.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-09T04:35:35.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-06-09T04:35:35.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-06-09T04:35:35.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-06-09T04:35:35.761 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:36.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:36.004 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:35:36.004 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:35:36.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.1 up 2023-06-09T04:35:36.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.1 master ceph-brx 2023-06-09T04:35:36.006 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:35:36.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:36.130 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-06-09T04:35:36.150 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-06-09T04:35:36.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:36.151 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-06-09T04:35:36.206 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-06-09T04:35:36.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:36.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:36.291 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:36.292 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:36.319 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:36.319 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:36.319 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:36.319 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 2@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:36.319 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:36.319 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:36.320 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:36.320 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:36.347 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:36.348 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:36.348 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:36.348 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 2@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:36.348 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:36.348 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:36.348 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:36.348 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:36.349 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-06-09T04:35:36.349 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-06-09T04:35:36.349 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=064bde79-2943-47e5-b941-05f234bd883f" will pass to kernel 2023-06-09T04:35:36.352 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-09T04:35:36.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:36.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-06-09T04:35:36.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:36.386 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:36.458 INFO:teuthology.orchestra.run.smithi092.stdout:client4683 2023-06-09T04:35:36.458 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:36.459 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:36.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:36.515 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:36.586 INFO:teuthology.orchestra.run.smithi092.stdout:client4683 2023-06-09T04:35:36.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:36.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:36.658 INFO:teuthology.orchestra.run.smithi092.stdout:client4683 2023-06-09T04:35:36.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:36.659 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/064bde79-2943-47e5-b941-05f234bd883f.client4683/status') 2023-06-09T04:35:36.738 INFO:teuthology.orchestra.run.smithi092.stdout:instance: client.4683 (3)192.168.0.2:0/1647629822 2023-06-09T04:35:36.738 INFO:teuthology.orchestra.run.smithi092.stdout:blocklisted: false 2023-06-09T04:35:36.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records in 2023-06-09T04:35:36.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+1 records out 2023-06-09T04:35:36.739 INFO:teuthology.orchestra.run.smithi092.stderr:69 bytes copied, 1.9141e-05 s, 3.6 MB/s 2023-06-09T04:35:36.740 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-06-09T04:35:36.740 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-09T04:35:36.740 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-09T04:35:36.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-09T04:35:36.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.898+0000 7fb682194700 1 Processor -- start 2023-06-09T04:35:36.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.898+0000 7fb682194700 1 -- start start 2023-06-09T04:35:36.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.899+0000 7fb682194700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb67c0aeaa0 0x7fb67c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:36.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.899+0000 7fb682194700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0adde0 con 0x7fb67c0b1360 2023-06-09T04:35:36.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.899+0000 7fb682194700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0af0f0 con 0x7fb67c0ae630 2023-06-09T04:35:36.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.899+0000 7fb682194700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0aeee0 con 0x7fb67c0aeaa0 2023-06-09T04:35:36.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.899+0000 7fb67bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb67c0aeaa0 0x7fb67c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:36.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.899+0000 7fb67b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb67c0b1360 0x7fb67c0b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36636/0 (socket says 172.21.15.92:36636) 2023-06-09T04:35:36.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.899+0000 7fb67b7fe700 1 -- 172.21.15.92:0/274180291 learned_addr learned my addr 172.21.15.92:0/274180291 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:36.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67a7fc700 1 -- 172.21.15.92:0/274180291 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771954299 0 0) 0x7fb67c0adde0 con 0x7fb67c0b1360 2023-06-09T04:35:36.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67a7fc700 1 -- 172.21.15.92:0/274180291 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6680035b0 con 0x7fb67c0b1360 2023-06-09T04:35:36.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67bfff700 1 -- 172.21.15.92:0/274180291 >> v1:172.21.15.123:6789/0 conn(0x7fb67c0ae630 legacy=0x7fb67c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:36.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67bfff700 1 -- 172.21.15.92:0/274180291 >> v1:172.21.15.92:6789/0 conn(0x7fb67c0b1360 legacy=0x7fb67c0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:36.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67bfff700 1 -- 172.21.15.92:0/274180291 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0ad4b0 con 0x7fb67c0aeaa0 2023-06-09T04:35:36.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67bfff700 1 --2- 172.21.15.92:0/274180291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0cdef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ddbb92c715e3a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:36.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67a7fc700 1 -- 172.21.15.92:0/274180291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400e070 con 0x7fb67c0aeaa0 2023-06-09T04:35:36.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67a7fc700 1 -- 172.21.15.92:0/274180291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400bb20 con 0x7fb67c0aeaa0 2023-06-09T04:35:36.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.900+0000 7fb67a7fc700 1 -- 172.21.15.92:0/274180291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674003a20 con 0x7fb67c0aeaa0 2023-06-09T04:35:36.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.901+0000 7fb682194700 1 -- 172.21.15.92:0/274180291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0aeaa0 msgr2=0x7fb67c0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:36.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.901+0000 7fb682194700 1 --2- 172.21.15.92:0/274180291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0cdef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:36.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.901+0000 7fb682194700 1 -- 172.21.15.92:0/274180291 shutdown_connections 2023-06-09T04:35:36.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.901+0000 7fb682194700 1 --2- 172.21.15.92:0/274180291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0cdef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:36.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.901+0000 7fb682194700 1 -- 172.21.15.92:0/274180291 >> 172.21.15.92:0/274180291 conn(0x7fb67c098c10 msgr2=0x7fb67c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:36.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.901+0000 7fb682194700 1 -- 172.21.15.92:0/274180291 shutdown_connections 2023-06-09T04:35:36.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.901+0000 7fb682194700 1 -- 172.21.15.92:0/274180291 wait complete. 2023-06-09T04:35:36.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.902+0000 7fb682194700 1 Processor -- start 2023-06-09T04:35:36.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.902+0000 7fb682194700 1 -- start start 2023-06-09T04:35:36.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb682194700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0ae630 0x7fb67c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:36.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb682194700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:36.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb682194700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb67c0b1360 0x7fb67c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:36.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb682194700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0ca570 con 0x7fb67c0ae630 2023-06-09T04:35:36.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb682194700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0c5fc0 con 0x7fb67c0aeaa0 2023-06-09T04:35:36.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb682194700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0c62a0 con 0x7fb67c0b1360 2023-06-09T04:35:36.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0ae630 0x7fb67c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:36.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0ae630 0x7fb67c0cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38310/0 (socket says 172.21.15.92:38310) 2023-06-09T04:35:36.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 -- 172.21.15.92:0/687484839 learned_addr learned my addr 172.21.15.92:0/687484839 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:36.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67bfff700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb67c0b1360 0x7fb67c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:36.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67affd700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:36.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 -- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb67c0b1360 msgr2=0x7fb67c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:36.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb67c0b1360 0x7fb67c0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:36.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 -- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c0aeaa0 msgr2=0x7fb67c0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:36.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:36.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 -- 172.21.15.92:0/687484839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674009a70 con 0x7fb67c0ae630 2023-06-09T04:35:36.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67affd700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:36.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb67b7fe700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0ae630 0x7fb67c0cc850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb664001260 tx=0x7fb66400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:36.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.903+0000 7fb673fff700 1 -- 172.21.15.92:0/687484839 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640074e0 con 0x7fb67c0ae630 2023-06-09T04:35:36.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.904+0000 7fb673fff700 1 -- 172.21.15.92:0/687484839 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb664007640 con 0x7fb67c0ae630 2023-06-09T04:35:36.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.904+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c0c9f20 con 0x7fb67c0ae630 2023-06-09T04:35:36.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.904+0000 7fb673fff700 1 -- 172.21.15.92:0/687484839 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664005a50 con 0x7fb67c0ae630 2023-06-09T04:35:36.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.904+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c0b47e0 con 0x7fb67c0ae630 2023-06-09T04:35:36.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.905+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c0740e0 con 0x7fb67c0ae630 2023-06-09T04:35:36.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.905+0000 7fb673fff700 1 -- 172.21.15.92:0/687484839 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb664005bb0 con 0x7fb67c0ae630 2023-06-09T04:35:36.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.905+0000 7fb673fff700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb65c06ad10 0x7fb65c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:36.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.906+0000 7fb67affd700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb65c06ad10 0x7fb65c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:36.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.906+0000 7fb673fff700 1 -- 172.21.15.92:0/687484839 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6640888c0 con 0x7fb67c0ae630 2023-06-09T04:35:36.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.906+0000 7fb67affd700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb65c06ad10 0x7fb65c06d1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0aff90 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:36.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:36.910+0000 7fb673fff700 1 -- 172.21.15.92:0/687484839 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb664053a00 con 0x7fb67c0ae630 2023-06-09T04:35:37.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.141+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb67c0c3160 con 0x7fb67c0ae630 2023-06-09T04:35:37.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.141+0000 7fb673fff700 1 -- 172.21.15.92:0/687484839 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb664017070 con 0x7fb67c0ae630 2023-06-09T04:35:37.143 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-09T04:35:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb65c06ad10 msgr2=0x7fb65c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb65c06ad10 0x7fb65c06d1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0aff90 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2023-06-09T04:35:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0ae630 msgr2=0x7fb67c0cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0ae630 0x7fb67c0cc850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb664001260 tx=0x7fb66400c860 comp rx=0 tx=0).stop 2023-06-09T04:35:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 shutdown_connections 2023-06-09T04:35:37.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb65c06ad10 0x7fb65c06d1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:37.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb67c0b1360 0x7fb67c0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:37.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb67c0ae630 0x7fb67c0cc850 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:37.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 --2- 172.21.15.92:0/687484839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c0aeaa0 0x7fb67c0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:37.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 >> 172.21.15.92:0/687484839 conn(0x7fb67c098c10 msgr2=0x7fb67c0b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:37.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 shutdown_connections 2023-06-09T04:35:37.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:37.143+0000 7fb682194700 1 -- 172.21.15.92:0/687484839 wait complete. 2023-06-09T04:35:37.157 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-09T04:35:37.157 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-09T04:35:37.157 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-06-09T04:35:37.157 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-06-09T04:35:37.157 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-09T04:35:37.158 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-09T04:35:37.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-09T04:35:37.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-09T04:35:37.158 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-09T04:35:37.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.159 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-06-09T04:35:37.177 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:37.178 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:37.178 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:37.178 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:37.178 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:37.178 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:37.178 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:37.179 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-09T04:35:37.180 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42472sec preferred_lft 42472sec 2023-06-09T04:35:37.180 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::4492:1ce0:179f:6ddd/64 scope link noprefixroute 2023-06-09T04:35:37.180 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:37.180 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-09T04:35:37.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.181 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:37.181 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:37.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2023-06-09T04:35:37.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2023-06-09T04:35:37.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2023-06-09T04:35:37.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-09T04:35:37.182 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:37.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.349 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-09T04:35:37.377 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-06-09T04:35:37.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.378 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-06-09T04:35:37.469 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-06-09T04:35:37.469 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-09T04:35:37.470 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-09T04:35:37.470 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-09T04:35:37.470 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-09T04:35:37.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.471 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:37.471 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:37.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-09T04:35:37.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-09T04:35:37.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-09T04:35:37.472 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:37.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.723 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-06-09T04:35:37.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.740 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-06-09T04:35:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.796 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:37.796 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:37.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-09T04:35:37.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-06-09T04:35:37.797 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:37.889 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-06-09T04:35:37.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:37.889 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:37.890 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:37.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-06-09T04:35:37.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-09T04:35:37.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-06-09T04:35:37.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-06-09T04:35:37.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-06-09T04:35:37.891 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:38.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:38.098 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:38.098 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:38.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2023-06-09T04:35:38.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2023-06-09T04:35:38.099 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:38.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:38.198 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-06-09T04:35:38.218 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-06-09T04:35:38.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:38.218 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-06-09T04:35:38.274 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-06-09T04:35:38.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:38.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:38.391 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:38.391 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:38.503 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 1@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:38.504 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:38.576 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:38.576 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:38.576 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:38.577 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 1@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:38.577 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:38.577 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:38.577 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:38.577 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:38.577 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-06-09T04:35:38.577 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-06-09T04:35:38.578 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=064bde79-2943-47e5-b941-05f234bd883f" will pass to kernel 2023-06-09T04:35:38.578 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-09T04:35:38.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:38.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-06-09T04:35:38.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:38.605 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:38.676 INFO:teuthology.orchestra.run.smithi123.stdout:client4695 2023-06-09T04:35:38.676 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:38.677 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:38.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:38.733 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:38.804 INFO:teuthology.orchestra.run.smithi123.stdout:client4695 2023-06-09T04:35:38.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:38.805 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:38.875 INFO:teuthology.orchestra.run.smithi123.stdout:client4695 2023-06-09T04:35:38.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:38.876 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/064bde79-2943-47e5-b941-05f234bd883f.client4695/status') 2023-06-09T04:35:38.955 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.4695 (3)172.21.15.123:0/4169993016 2023-06-09T04:35:38.955 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2023-06-09T04:35:38.956 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2023-06-09T04:35:38.956 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2023-06-09T04:35:38.957 INFO:teuthology.orchestra.run.smithi123.stderr:71 bytes copied, 2.2039e-05 s, 3.2 MB/s 2023-06-09T04:35:38.957 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-06-09T04:35:38.957 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-09T04:35:38.957 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-09T04:35:38.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-09T04:35:39.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.088+0000 7f787b4a3700 1 Processor -- start 2023-06-09T04:35:39.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.088+0000 7f787b4a3700 1 -- start start 2023-06-09T04:35:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f787b4a3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78740cf010 0x7f78740ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f787b4a3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740ac400 0x7f787406e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f787b4a3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78740ce730 con 0x7f787415c7d0 2023-06-09T04:35:39.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f787b4a3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787406aaf0 con 0x7f78740cf010 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f787b4a3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787415cba0 con 0x7f78740ac400 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f7878a3e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78740cf010 0x7f78740ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f787923f700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f787415c7d0 0x7f78740ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59146/0 (socket says 172.21.15.92:59146) 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f7878a3e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f78740cf010 0x7f78740ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58728/0 (socket says 172.21.15.92:58728) 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f787923f700 1 -- 172.21.15.92:0/1838437525 learned_addr learned my addr 172.21.15.92:0/1838437525 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.089+0000 7f7879a40700 1 --2- 172.21.15.92:0/1838437525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740ac400 0x7f787406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f7879a40700 1 -- 172.21.15.92:0/1838437525 >> v2:172.21.15.123:3300/0 conn(0x7f78740cf010 msgr2=0x7f78740ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f7879a40700 1 --2- 172.21.15.92:0/1838437525 >> v2:172.21.15.123:3300/0 conn(0x7f78740cf010 0x7f78740ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f7879a40700 1 -- 172.21.15.92:0/1838437525 >> v1:172.21.15.123:6789/0 conn(0x7f787415c7d0 legacy=0x7f78740ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f7879a40700 1 -- 172.21.15.92:0/1838437525 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874069ee0 con 0x7f78740ac400 2023-06-09T04:35:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f786bfff700 1 -- 172.21.15.92:0/1838437525 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731709479 0 0) 0x7f7864001160 con 0x7f787415c7d0 2023-06-09T04:35:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f7878a3e700 1 --2- 172.21.15.92:0/1838437525 >> v2:172.21.15.123:3300/0 conn(0x7f78740cf010 0x7f78740ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f7879a40700 1 --2- 172.21.15.92:0/1838437525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740ac400 0x7f787406e220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f786c009d90 tx=0x7f786c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54b69c71ef0c363 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f786bfff700 1 -- 172.21.15.92:0/1838437525 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00e070 con 0x7f78740ac400 2023-06-09T04:35:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f786bfff700 1 -- 172.21.15.92:0/1838437525 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00bb20 con 0x7f78740ac400 2023-06-09T04:35:39.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f786bfff700 1 -- 172.21.15.92:0/1838437525 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c012430 con 0x7f78740ac400 2023-06-09T04:35:39.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f787b4a3700 1 -- 172.21.15.92:0/1838437525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740ac400 msgr2=0x7f787406e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:39.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f787b4a3700 1 --2- 172.21.15.92:0/1838437525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740ac400 0x7f787406e220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f786c009d90 tx=0x7f786c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:39.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.090+0000 7f787b4a3700 1 -- 172.21.15.92:0/1838437525 shutdown_connections 2023-06-09T04:35:39.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.091+0000 7f787b4a3700 1 --2- 172.21.15.92:0/1838437525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f78740ac400 0x7f787406e220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.091+0000 7f787b4a3700 1 --2- 172.21.15.92:0/1838437525 >> v2:172.21.15.123:3300/0 conn(0x7f78740cf010 0x7f78740ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.091+0000 7f787b4a3700 1 -- 172.21.15.92:0/1838437525 >> 172.21.15.92:0/1838437525 conn(0x7f7874098c10 msgr2=0x7f78740cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:39.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.091+0000 7f787b4a3700 1 -- 172.21.15.92:0/1838437525 shutdown_connections 2023-06-09T04:35:39.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.091+0000 7f787b4a3700 1 -- 172.21.15.92:0/1838437525 wait complete. 2023-06-09T04:35:39.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.092+0000 7f787b4a3700 1 Processor -- start 2023-06-09T04:35:39.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.092+0000 7f787b4a3700 1 -- start start 2023-06-09T04:35:39.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.092+0000 7f787b4a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ac400 0x7f78740cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:39.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.092+0000 7f787b4a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740cf010 0x7f78740b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:39.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.092+0000 7f787b4a3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f787415c7d0 0x7f78740b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:39.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f787b4a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740cc610 con 0x7f78740cf010 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f787b4a3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740cb990 con 0x7f78740ac400 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f787b4a3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740cbc70 con 0x7f787415c7d0 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740cf010 0x7f78740b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f787923f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ac400 0x7f78740cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740cf010 0x7f78740b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38326/0 (socket says 172.21.15.92:38326) 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f787923f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ac400 0x7f78740cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:58732/0 (socket says 172.21.15.92:58732) 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 -- 172.21.15.92:0/327517805 learned_addr learned my addr 172.21.15.92:0/327517805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:39.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7879a40700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f787415c7d0 0x7f78740b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 -- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f787415c7d0 msgr2=0x7f78740b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f787415c7d0 0x7f78740b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 -- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ac400 msgr2=0x7f78740cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ac400 0x7f78740cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 -- 172.21.15.92:0/327517805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700094a0 con 0x7f78740cf010 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f787923f700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ac400 0x7f78740cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7878a3e700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740cf010 0x7f78740b0750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f787000ebe0 tx=0x7f787000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:39.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7869ffb700 1 -- 172.21.15.92:0/327517805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78700097a0 con 0x7f78740cf010 2023-06-09T04:35:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7869ffb700 1 -- 172.21.15.92:0/327517805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787000c850 con 0x7f78740cf010 2023-06-09T04:35:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c009a70 con 0x7f78740cf010 2023-06-09T04:35:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.093+0000 7f7869ffb700 1 -- 172.21.15.92:0/327517805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000cbe0 con 0x7f78740cf010 2023-06-09T04:35:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.094+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78740cc230 con 0x7f78740cf010 2023-06-09T04:35:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.094+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78740740e0 con 0x7f78740cf010 2023-06-09T04:35:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.095+0000 7f7869ffb700 1 -- 172.21.15.92:0/327517805 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f787000cd40 con 0x7f78740cf010 2023-06-09T04:35:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.095+0000 7f7869ffb700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f785806ace0 0x7f785806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.095+0000 7f7869ffb700 1 -- 172.21.15.92:0/327517805 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7870089230 con 0x7f78740cf010 2023-06-09T04:35:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.095+0000 7f787923f700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f785806ace0 0x7f785806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:39.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.096+0000 7f787923f700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f785806ace0 0x7f785806d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7864000fb0 tx=0x7f7864007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:39.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.100+0000 7f7869ffb700 1 -- 172.21.15.92:0/327517805 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7870054250 con 0x7f78740cf010 2023-06-09T04:35:39.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.332+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78740b0110 con 0x7f78740cf010 2023-06-09T04:35:39.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.333+0000 7f7869ffb700 1 -- 172.21.15.92:0/327517805 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7870057870 con 0x7f78740cf010 2023-06-09T04:35:39.334 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f785806ace0 msgr2=0x7f785806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f785806ace0 0x7f785806d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7864000fb0 tx=0x7f7864007ec0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740cf010 msgr2=0x7f78740b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740cf010 0x7f78740b0750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f787000ebe0 tx=0x7f787000eef0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 shutdown_connections 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f785806ace0 0x7f785806d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f787415c7d0 0x7f78740b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78740ac400 0x7f78740cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 --2- 172.21.15.92:0/327517805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78740cf010 0x7f78740b0750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 >> 172.21.15.92:0/327517805 conn(0x7f7874098c10 msgr2=0x7f78740ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 shutdown_connections 2023-06-09T04:35:39.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:39.335+0000 7f787b4a3700 1 -- 172.21.15.92:0/327517805 wait complete. 2023-06-09T04:35:39.348 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-09T04:35:39.348 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-09T04:35:39.348 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-06-09T04:35:39.348 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-06-09T04:35:39.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-09T04:35:39.349 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-09T04:35:39.349 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-09T04:35:39.349 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-09T04:35:39.349 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-06-09T04:35:39.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.350 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-06-09T04:35:39.367 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:39.367 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:39.368 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:39.368 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.368 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:39.368 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.368 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:39.368 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:39.368 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42469sec preferred_lft 42469sec 2023-06-09T04:35:39.369 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::4492:1ce0:179f:6ddd/64 scope link noprefixroute 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5a:4f:31:ad:c6:bf brd ff:ff:ff:ff:ff:ff 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::4cf:38ff:fec4:3989/64 scope link 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.370 INFO:teuthology.orchestra.run.smithi123.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-09T04:35:39.371 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5a:4f:31:ad:c6:bf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-09T04:35:39.371 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::584f:31ff:fead:c6bf/64 scope link tentative 2023-06-09T04:35:39.371 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.371 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-09T04:35:39.432 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-06-09T04:35:39.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.434 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-06-09T04:35:39.492 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-06-09T04:35:39.493 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-09T04:35:39.493 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-09T04:35:39.493 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-09T04:35:39.493 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-09T04:35:39.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.494 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:39.494 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:39.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-09T04:35:39.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-09T04:35:39.495 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-09T04:35:39.495 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:39.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.597 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-06-09T04:35:39.614 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-09T04:35:39.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.615 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-06-09T04:35:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-06-09T04:35:39.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.671 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:39.671 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:39.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-09T04:35:39.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-06-09T04:35:39.672 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:39.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-09T04:35:39.804 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.805 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-09T04:35:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether c6:88:56:d2:a9:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-09T04:35:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-09T04:35:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::c488:56ff:fed2:a905/64 scope link 2023-06-09T04:35:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-09T04:35:39.897 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:35:39.897 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:35:39.897 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:35:39.897 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.897 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-06-09T04:35:39.897 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.898 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-09T04:35:39.898 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether c6:88:56:d2:a9:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-09T04:35:39.898 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-09T04:35:39.898 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.898 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::c488:56ff:fed2:a905/64 scope link 2023-06-09T04:35:39.898 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:35:39.910 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-06-09T04:35:39.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:39.911 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:39.911 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:39.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-06-09T04:35:39.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-09T04:35:39.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-06-09T04:35:39.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-06-09T04:35:39.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-06-09T04:35:39.912 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:40.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:40.151 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-06-09T04:35:40.151 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-06-09T04:35:40.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2023-06-09T04:35:40.152 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2023-06-09T04:35:40.152 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-06-09T04:35:40.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:40.257 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-06-09T04:35:40.276 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-06-09T04:35:40.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:35:40.277 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-06-09T04:35:40.333 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-06-09T04:35:40.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:40.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:40.418 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:40.418 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:40.449 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:40.449 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:40.449 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:40.449 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 3@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:40.450 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:40.450 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:40.450 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:40.450 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:40.475 INFO:teuthology.orchestra.run.smithi123.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:35:40.475 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:35:40.475 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:35:40.475 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with new device syntax: 3@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-06-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-06-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi123.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=064bde79-2943-47e5-b941-05f234bd883f" will pass to kernel 2023-06-09T04:35:40.478 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-09T04:35:40.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:40.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-06-09T04:35:40.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:40.507 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:40.578 INFO:teuthology.orchestra.run.smithi123.stdout:client4582 2023-06-09T04:35:40.578 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:40.579 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:40.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:40.635 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:40.706 INFO:teuthology.orchestra.run.smithi123.stdout:client4582 2023-06-09T04:35:40.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:40.707 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:35:40.777 INFO:teuthology.orchestra.run.smithi123.stdout:client4582 2023-06-09T04:35:40.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:40.778 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/064bde79-2943-47e5-b941-05f234bd883f.client4582/status') 2023-06-09T04:35:40.856 INFO:teuthology.orchestra.run.smithi123.stdout:instance: client.4582 (3)192.168.0.2:0/2337345221 2023-06-09T04:35:40.857 INFO:teuthology.orchestra.run.smithi123.stdout:blocklisted: false 2023-06-09T04:35:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records in 2023-06-09T04:35:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:0+1 records out 2023-06-09T04:35:40.857 INFO:teuthology.orchestra.run.smithi123.stderr:69 bytes copied, 2.5515e-05 s, 2.7 MB/s 2023-06-09T04:35:40.858 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-06-09T04:35:40.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:41.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.007+0000 7f8498f14700 1 Processor -- start 2023-06-09T04:35:41.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.008+0000 7f8498f14700 1 -- start start 2023-06-09T04:35:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.008+0000 7f8498f14700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84940ac400 0x7f84940ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.008+0000 7f8498f14700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84940ce730 con 0x7f849415c7d0 2023-06-09T04:35:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.008+0000 7f8498f14700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849406aaf0 con 0x7f84940cf010 2023-06-09T04:35:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.008+0000 7f8498f14700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849415cba0 con 0x7f84940ac400 2023-06-09T04:35:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f849415c7d0 0x7f84940ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36664/0 (socket says 172.21.15.92:36664) 2023-06-09T04:35:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849259c700 1 -- 172.21.15.92:0/1345014239 learned_addr learned my addr 172.21.15.92:0/1345014239 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:41.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f8492d9d700 1 --2- 172.21.15.92:0/1345014239 >> v2:172.21.15.123:3300/0 conn(0x7f84940ac400 0x7f84940ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266297690 0 0) 0x7f8484001290 con 0x7f849415c7d0 2023-06-09T04:35:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f847c0035b0 con 0x7f849415c7d0 2023-06-09T04:35:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035685772 0 0) 0x7f849406aaf0 con 0x7f84940cf010 2023-06-09T04:35:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8484001290 con 0x7f84940cf010 2023-06-09T04:35:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f8492d9d700 1 -- 172.21.15.92:0/1345014239 >> v1:172.21.15.123:6789/0 conn(0x7f84940cf010 legacy=0x7f849406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1547089862 0 0) 0x7f847c0035b0 con 0x7f849415c7d0 2023-06-09T04:35:41.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f8492d9d700 1 -- 172.21.15.92:0/1345014239 >> v1:172.21.15.92:6789/0 conn(0x7f849415c7d0 legacy=0x7f84940ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f8492d9d700 1 -- 172.21.15.92:0/1345014239 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8494069ee0 con 0x7f84940ac400 2023-06-09T04:35:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f8492d9d700 1 --2- 172.21.15.92:0/1345014239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 0x7f84940ce1f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f848c009d90 tx=0x7f848c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6978d66d52d5d42 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.009+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c00e070 con 0x7f84940ac400 2023-06-09T04:35:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c003a20 con 0x7f84940ac400 2023-06-09T04:35:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f849159a700 1 -- 172.21.15.92:0/1345014239 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484001980 con 0x7f84940ac400 2023-06-09T04:35:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f8498f14700 1 -- 172.21.15.92:0/1345014239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 msgr2=0x7f84940ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f8498f14700 1 --2- 172.21.15.92:0/1345014239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 0x7f84940ce1f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f848c009d90 tx=0x7f848c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f8498f14700 1 -- 172.21.15.92:0/1345014239 shutdown_connections 2023-06-09T04:35:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f8498f14700 1 --2- 172.21.15.92:0/1345014239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 0x7f84940ce1f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f8498f14700 1 -- 172.21.15.92:0/1345014239 >> 172.21.15.92:0/1345014239 conn(0x7f8494098c10 msgr2=0x7f84940cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f8498f14700 1 -- 172.21.15.92:0/1345014239 shutdown_connections 2023-06-09T04:35:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.010+0000 7f8498f14700 1 -- 172.21.15.92:0/1345014239 wait complete. 2023-06-09T04:35:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 Processor -- start 2023-06-09T04:35:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 -- start start 2023-06-09T04:35:41.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 0x7f84940b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84940cf010 0x7f84940c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849415c7d0 0x7f84941554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84940ca3a0 con 0x7f849415c7d0 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84940ca640 con 0x7f84940ac400 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8498f14700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84940ca920 con 0x7f84940cf010 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8492d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849415c7d0 0x7f84941554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8492d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849415c7d0 0x7f84941554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38340/0 (socket says 172.21.15.92:38340) 2023-06-09T04:35:41.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.012+0000 7f8492d9d700 1 -- 172.21.15.92:0/1371116381 learned_addr learned my addr 172.21.15.92:0/1371116381 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f849259c700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 0x7f84940b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8491d9b700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84940cf010 0x7f84940c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8492d9d700 1 -- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84940cf010 msgr2=0x7f84940c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8492d9d700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84940cf010 0x7f84940c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8492d9d700 1 -- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 msgr2=0x7f84940b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8492d9d700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 0x7f84940b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8492d9d700 1 -- 172.21.15.92:0/1371116381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009a70 con 0x7f849415c7d0 2023-06-09T04:35:41.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8492d9d700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849415c7d0 0x7f84941554b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b370 tx=0x7f848c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8482ffd700 1 -- 172.21.15.92:0/1371116381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c004440 con 0x7f849415c7d0 2023-06-09T04:35:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84940cac00 con 0x7f849415c7d0 2023-06-09T04:35:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8482ffd700 1 -- 172.21.15.92:0/1371116381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c0045a0 con 0x7f849415c7d0 2023-06-09T04:35:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8482ffd700 1 -- 172.21.15.92:0/1371116381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c004950 con 0x7f849415c7d0 2023-06-09T04:35:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.013+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84940cb1c0 con 0x7f849415c7d0 2023-06-09T04:35:41.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.014+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84940740e0 con 0x7f849415c7d0 2023-06-09T04:35:41.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.014+0000 7f8482ffd700 1 -- 172.21.15.92:0/1371116381 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f848c01d470 con 0x7f849415c7d0 2023-06-09T04:35:41.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.015+0000 7f8482ffd700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f847406ace0 0x7f847406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.015+0000 7f8482ffd700 1 -- 172.21.15.92:0/1371116381 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f848c08afb0 con 0x7f849415c7d0 2023-06-09T04:35:41.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.015+0000 7f849259c700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f847406ace0 0x7f847406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.015+0000 7f849259c700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f847406ace0 0x7f847406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8484001c80 tx=0x7f8484001bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.019+0000 7f8482ffd700 1 -- 172.21.15.92:0/1371116381 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f848c0bd310 con 0x7f849415c7d0 2023-06-09T04:35:41.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.211+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f849406f010 con 0x7f849415c7d0 2023-06-09T04:35:41.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.212+0000 7f8482ffd700 1 -- 172.21.15.92:0/1371116381 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f848c059b00 con 0x7f849415c7d0 2023-06-09T04:35:41.214 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:41.214 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:41.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f847406ace0 msgr2=0x7f847406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f847406ace0 0x7f847406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8484001c80 tx=0x7f8484001bf0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849415c7d0 msgr2=0x7f84941554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849415c7d0 0x7f84941554b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f848c00b370 tx=0x7f848c004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 shutdown_connections 2023-06-09T04:35:41.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f847406ace0 0x7f847406d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849415c7d0 0x7f84941554b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84940ac400 0x7f84940b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 --2- 172.21.15.92:0/1371116381 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84940cf010 0x7f84940c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 >> 172.21.15.92:0/1371116381 conn(0x7f8494098c10 msgr2=0x7f84940b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 shutdown_connections 2023-06-09T04:35:41.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.214+0000 7f8498f14700 1 -- 172.21.15.92:0/1371116381 wait complete. 2023-06-09T04:35:41.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:41.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.351+0000 7ff986cfc700 1 Processor -- start 2023-06-09T04:35:41.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.351+0000 7ff986cfc700 1 -- start start 2023-06-09T04:35:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff986cfc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9800cf440 0x7ff98015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff986cfc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98015d040 con 0x7ff9800cce60 2023-06-09T04:35:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff986cfc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff980160ab0 con 0x7ff980069db0 2023-06-09T04:35:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff986cfc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98015ca10 con 0x7ff9800cf440 2023-06-09T04:35:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff97ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9800cf440 0x7ff98015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff985299700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff9800cce60 0x7ff98006ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36684/0 (socket says 172.21.15.92:36684) 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff97ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9800cf440 0x7ff98015d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38348/0 (socket says 172.21.15.92:38348) 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff985299700 1 -- 172.21.15.92:0/1551322213 learned_addr learned my addr 172.21.15.92:0/1551322213 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff97ffff700 1 -- 172.21.15.92:0/1551322213 >> v1:172.21.15.123:6789/0 conn(0x7ff980069db0 legacy=0x7ff98006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff97ffff700 1 -- 172.21.15.92:0/1551322213 >> v1:172.21.15.92:6789/0 conn(0x7ff9800cce60 legacy=0x7ff98006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff97ffff700 1 -- 172.21.15.92:0/1551322213 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980160d50 con 0x7ff9800cf440 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff97f7fe700 1 -- 172.21.15.92:0/1551322213 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464116306 0 0) 0x7ff98015d040 con 0x7ff9800cce60 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.352+0000 7ff97ffff700 1 --2- 172.21.15.92:0/1551322213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9800cf440 0x7ff98015d4d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009840 tx=0x7ff96c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a0074659fe9878 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff97f7fe700 1 -- 172.21.15.92:0/1551322213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c011830 con 0x7ff9800cf440 2023-06-09T04:35:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff97f7fe700 1 -- 172.21.15.92:0/1551322213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96c011990 con 0x7ff9800cf440 2023-06-09T04:35:41.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff97f7fe700 1 -- 172.21.15.92:0/1551322213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96c011d20 con 0x7ff9800cf440 2023-06-09T04:35:41.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff986cfc700 1 -- 172.21.15.92:0/1551322213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9800cf440 msgr2=0x7ff98015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff986cfc700 1 --2- 172.21.15.92:0/1551322213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9800cf440 0x7ff98015d4d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009840 tx=0x7ff96c009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff986cfc700 1 -- 172.21.15.92:0/1551322213 shutdown_connections 2023-06-09T04:35:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff986cfc700 1 --2- 172.21.15.92:0/1551322213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9800cf440 0x7ff98015d4d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff986cfc700 1 -- 172.21.15.92:0/1551322213 >> 172.21.15.92:0/1551322213 conn(0x7ff9800cc380 msgr2=0x7ff9800cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff986cfc700 1 -- 172.21.15.92:0/1551322213 shutdown_connections 2023-06-09T04:35:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.353+0000 7ff986cfc700 1 -- 172.21.15.92:0/1551322213 wait complete. 2023-06-09T04:35:41.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.354+0000 7ff986cfc700 1 Processor -- start 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.354+0000 7ff986cfc700 1 -- start start 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff986cfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff980069db0 0x7ff9801f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff986cfc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9800cce60 0x7ff980155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff986cfc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9800cf440 0x7ff9800af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff986cfc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800c8c70 con 0x7ff980069db0 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff986cfc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9801f5ac0 con 0x7ff9800cf440 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff986cfc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff980155a50 con 0x7ff9800cce60 2023-06-09T04:35:41.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff980069db0 0x7ff9801f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff980069db0 0x7ff9801f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38354/0 (socket says 172.21.15.92:38354) 2023-06-09T04:35:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 -- 172.21.15.92:0/7950967 learned_addr learned my addr 172.21.15.92:0/7950967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 -- 172.21.15.92:0/7950967 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9800cce60 msgr2=0x7ff980155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9800cce60 0x7ff980155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 -- 172.21.15.92:0/7950967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9800cf440 msgr2=0x7ff9800af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9800cf440 0x7ff9800af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.355+0000 7ff984a98700 1 -- 172.21.15.92:0/7950967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0094a0 con 0x7ff980069db0 2023-06-09T04:35:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.356+0000 7ff984a98700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff980069db0 0x7ff9801f6510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff97400c4e0 tx=0x7ff97400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.356+0000 7ff97d7fa700 1 -- 172.21.15.92:0/7950967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9740052d0 con 0x7ff980069db0 2023-06-09T04:35:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.356+0000 7ff97d7fa700 1 -- 172.21.15.92:0/7950967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff974005430 con 0x7ff980069db0 2023-06-09T04:35:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.356+0000 7ff97d7fa700 1 -- 172.21.15.92:0/7950967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9740038c0 con 0x7ff980069db0 2023-06-09T04:35:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.356+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9800afca0 con 0x7ff980069db0 2023-06-09T04:35:41.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.356+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9801f5d10 con 0x7ff980069db0 2023-06-09T04:35:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.357+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98006be50 con 0x7ff980069db0 2023-06-09T04:35:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.357+0000 7ff97d7fa700 1 -- 172.21.15.92:0/7950967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff974003a20 con 0x7ff980069db0 2023-06-09T04:35:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.358+0000 7ff97d7fa700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff96406ace0 0x7ff96406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.358+0000 7ff97d7fa700 1 -- 172.21.15.92:0/7950967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff974088910 con 0x7ff980069db0 2023-06-09T04:35:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.358+0000 7ff97ffff700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff96406ace0 0x7ff96406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.358+0000 7ff97ffff700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff96406ace0 0x7ff96406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9800c9bd0 tx=0x7ff96c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.362+0000 7ff97d7fa700 1 -- 172.21.15.92:0/7950967 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff974053a50 con 0x7ff980069db0 2023-06-09T04:35:41.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.553+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff98004e8e0 con 0x7ff980069db0 2023-06-09T04:35:41.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.553+0000 7ff97d7fa700 1 -- 172.21.15.92:0/7950967 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff974017020 con 0x7ff980069db0 2023-06-09T04:35:41.555 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff96406ace0 msgr2=0x7ff96406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff96406ace0 0x7ff96406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9800c9bd0 tx=0x7ff96c010000 comp rx=0 tx=0).stop 2023-06-09T04:35:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff980069db0 msgr2=0x7ff9801f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff980069db0 0x7ff9801f6510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff97400c4e0 tx=0x7ff97400c8a0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 shutdown_connections 2023-06-09T04:35:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff96406ace0 0x7ff96406d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9800cf440 0x7ff9800af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff980069db0 0x7ff9801f6510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 --2- 172.21.15.92:0/7950967 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff9800cce60 0x7ff980155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 >> 172.21.15.92:0/7950967 conn(0x7ff9800cc380 msgr2=0x7ff9800cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:41.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 shutdown_connections 2023-06-09T04:35:41.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.555+0000 7ff986cfc700 1 -- 172.21.15.92:0/7950967 wait complete. 2023-06-09T04:35:41.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:41.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.689+0000 7f6c90764700 1 Processor -- start 2023-06-09T04:35:41.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.689+0000 7f6c90764700 1 -- start start 2023-06-09T04:35:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.690+0000 7f6c90764700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6c880b60d0 0x7f6c880d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.690+0000 7f6c90764700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 0x7f6c880ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.690+0000 7f6c90764700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c880cb700 con 0x7f6c8815e810 2023-06-09T04:35:41.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.690+0000 7f6c90764700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c880b64a0 con 0x7f6c880b60d0 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.690+0000 7f6c90764700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880ccf50 con 0x7f6c880cda90 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.690+0000 7f6c8dcff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c8815e810 0x7f6c880cd380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36696/0 (socket says 172.21.15.92:36696) 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.690+0000 7f6c8dcff700 1 -- 172.21.15.92:0/2578158126 learned_addr learned my addr 172.21.15.92:0/2578158126 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8e500700 1 --2- 172.21.15.92:0/2578158126 >> v2:172.21.15.123:3300/0 conn(0x7f6c880b60d0 0x7f6c880d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8ed01700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 0x7f6c880ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226956006 0 0) 0x7f6c84001110 con 0x7f6c8815e810 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c740035b0 con 0x7f6c8815e810 2023-06-09T04:35:41.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464369271 0 0) 0x7f6c740035b0 con 0x7f6c8815e810 2023-06-09T04:35:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 msgr2=0x7f6c880ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 0x7f6c880ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 >> v2:172.21.15.123:3300/0 conn(0x7f6c880b60d0 msgr2=0x7f6c880d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 --2- 172.21.15.92:0/2578158126 >> v2:172.21.15.123:3300/0 conn(0x7f6c880b60d0 0x7f6c880d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880cb140 con 0x7f6c8815e810 2023-06-09T04:35:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f6c84001800 con 0x7f6c8815e810 2023-06-09T04:35:41.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 >> v1:172.21.15.92:6789/0 conn(0x7f6c8815e810 legacy=0x7f6c880cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 0x7f6c74003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74003f80 0x7f6c74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74004d90 0x7f6c74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880b64a0 con 0x7f6c74004d90 2023-06-09T04:35:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880ccf50 con 0x7f6c74003f80 2023-06-09T04:35:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7400ba10 con 0x7f6c880cda90 2023-06-09T04:35:41.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8ed01700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74004d90 0x7f6c74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8dcff700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74003f80 0x7f6c74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8e500700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 0x7f6c74003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8ed01700 1 -- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 msgr2=0x7f6c74003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8ed01700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c880cda90 0x7f6c74003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8ed01700 1 -- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74003f80 msgr2=0x7f6c74004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8ed01700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c74003f80 0x7f6c74004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.691+0000 7f6c8ed01700 1 -- 172.21.15.92:0/2578158126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c740035b0 con 0x7f6c74004d90 2023-06-09T04:35:41.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c8ed01700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74004d90 0x7f6c74005200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6c880c79f0 tx=0x7f6c74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed38508f63847d72 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000ebd0 con 0x7f6c74004d90 2023-06-09T04:35:41.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8000f040 con 0x7f6c74004d90 2023-06-09T04:35:41.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c8d4fe700 1 -- 172.21.15.92:0/2578158126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80014480 con 0x7f6c74004d90 2023-06-09T04:35:41.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c90764700 1 -- 172.21.15.92:0/2578158126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74004d90 msgr2=0x7f6c74005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c90764700 1 --2- 172.21.15.92:0/2578158126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c74004d90 0x7f6c74005200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6c880c79f0 tx=0x7f6c74005880 comp rx=0 tx=0).stop 2023-06-09T04:35:41.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c8ed01700 1 -- 172.21.15.92:0/2578158126 reap_dead start 2023-06-09T04:35:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c90764700 1 -- 172.21.15.92:0/2578158126 shutdown_connections 2023-06-09T04:35:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c90764700 1 -- 172.21.15.92:0/2578158126 >> 172.21.15.92:0/2578158126 conn(0x7f6c8806e4d0 msgr2=0x7f6c8806e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c90764700 1 -- 172.21.15.92:0/2578158126 shutdown_connections 2023-06-09T04:35:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.692+0000 7f6c90764700 1 -- 172.21.15.92:0/2578158126 wait complete. 2023-06-09T04:35:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.693+0000 7f6c90764700 1 Processor -- start 2023-06-09T04:35:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.693+0000 7f6c90764700 1 -- start start 2023-06-09T04:35:41.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c90764700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c74004d90 0x7f6c880c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c90764700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8815a850 0x7f6c8815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c90764700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c8815bb80 0x7f6c880b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c90764700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8806b2d0 con 0x7f6c8815a850 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c90764700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880c7800 con 0x7f6c8815bb80 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c90764700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8815c050 con 0x7f6c74004d90 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c8dcff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8815a850 0x7f6c8815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c8dcff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8815a850 0x7f6c8815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38378/0 (socket says 172.21.15.92:38378) 2023-06-09T04:35:41.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c8dcff700 1 -- 172.21.15.92:0/1905888652 learned_addr learned my addr 172.21.15.92:0/1905888652 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c8e500700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c74004d90 0x7f6c880c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.694+0000 7f6c8ed01700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c8815bb80 0x7f6c880b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c8dcff700 1 -- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c74004d90 msgr2=0x7f6c880c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c8dcff700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c74004d90 0x7f6c880c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c8dcff700 1 -- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c8815bb80 msgr2=0x7f6c880b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c8dcff700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c8815bb80 0x7f6c880b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c8dcff700 1 -- 172.21.15.92:0/1905888652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8815c330 con 0x7f6c8815a850 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c8dcff700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8815a850 0x7f6c8815b640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c84001c30 tx=0x7f6c840075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c7effd700 1 -- 172.21.15.92:0/1905888652 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400f690 con 0x7f6c8815a850 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c7effd700 1 -- 172.21.15.92:0/1905888652 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8400f830 con 0x7f6c8815a850 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8806af60 con 0x7f6c8815a850 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c7effd700 1 -- 172.21.15.92:0/1905888652 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400fbe0 con 0x7f6c8815a850 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.695+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c880b3ce0 con 0x7f6c8815a850 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.696+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8804e8e0 con 0x7f6c8815a850 2023-06-09T04:35:41.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.696+0000 7f6c7effd700 1 -- 172.21.15.92:0/1905888652 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c8400fd40 con 0x7f6c8815a850 2023-06-09T04:35:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.697+0000 7f6c7effd700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c6c06ace0 0x7f6c6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.697+0000 7f6c7effd700 1 -- 172.21.15.92:0/1905888652 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c840892e0 con 0x7f6c8815a850 2023-06-09T04:35:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.697+0000 7f6c8e500700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c6c06ace0 0x7f6c6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.697+0000 7f6c8e500700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c6c06ace0 0x7f6c6c06d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009ce0 tx=0x7f6c78009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:41.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.701+0000 7f6c7effd700 1 -- 172.21.15.92:0/1905888652 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6c84054390 con 0x7f6c8815a850 2023-06-09T04:35:41.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.888+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6c880cb700 con 0x7f6c8815a850 2023-06-09T04:35:41.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.888+0000 7f6c7effd700 1 -- 172.21.15.92:0/1905888652 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6c880cb700 con 0x7f6c8815a850 2023-06-09T04:35:41.890 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:41.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c6c06ace0 msgr2=0x7f6c6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c6c06ace0 0x7f6c6c06d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009ce0 tx=0x7f6c78009210 comp rx=0 tx=0).stop 2023-06-09T04:35:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8815a850 msgr2=0x7f6c8815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8815a850 0x7f6c8815b640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c84001c30 tx=0x7f6c840075f0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 shutdown_connections 2023-06-09T04:35:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c6c06ace0 0x7f6c6c06d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c8815bb80 0x7f6c880b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c74004d90 0x7f6c880c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 --2- 172.21.15.92:0/1905888652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c8815a850 0x7f6c8815b640 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 >> 172.21.15.92:0/1905888652 conn(0x7f6c8806e4d0 msgr2=0x7f6c88159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 shutdown_connections 2023-06-09T04:35:41.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:41.890+0000 7f6c90764700 1 -- 172.21.15.92:0/1905888652 wait complete. 2023-06-09T04:35:41.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:42.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.025+0000 7f292683a700 1 Processor -- start 2023-06-09T04:35:42.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.025+0000 7f292683a700 1 -- start start 2023-06-09T04:35:42.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f292683a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29200ce3d0 0x7f29200713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f292683a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29200aed50 0x7f29200ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f292683a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f292015d470 0x7f292015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f292683a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2920072350 con 0x7f292015d470 2023-06-09T04:35:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f292683a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200ce190 con 0x7f29200ce3d0 2023-06-09T04:35:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f292683a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200cece0 con 0x7f29200aed50 2023-06-09T04:35:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f292015d470 0x7f292015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f292015d470 0x7f292015d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38388/0 (socket says 172.21.15.92:38388) 2023-06-09T04:35:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 -- 172.21.15.92:0/734970805 learned_addr learned my addr 172.21.15.92:0/734970805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:42.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f291f7fe700 1 --2- 172.21.15.92:0/734970805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29200aed50 0x7f29200ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f291ffff700 1 --2- 172.21.15.92:0/734970805 >> v2:172.21.15.123:3300/0 conn(0x7f29200ce3d0 0x7f29200713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 -- 172.21.15.92:0/734970805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29200aed50 msgr2=0x7f29200ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 --2- 172.21.15.92:0/734970805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29200aed50 0x7f29200ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 -- 172.21.15.92:0/734970805 >> v2:172.21.15.123:3300/0 conn(0x7f29200ce3d0 msgr2=0x7f29200713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 --2- 172.21.15.92:0/734970805 >> v2:172.21.15.123:3300/0 conn(0x7f29200ce3d0 0x7f29200713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f2924dd7700 1 -- 172.21.15.92:0/734970805 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920071ab0 con 0x7f292015d470 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.026+0000 7f291f7fe700 1 --2- 172.21.15.92:0/734970805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29200aed50 0x7f29200ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:42.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f2924dd7700 1 --2- 172.21.15.92:0/734970805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f292015d470 0x7f292015d8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f292006fee0 tx=0x7f291800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29368710d11ce9f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f291effd700 1 -- 172.21.15.92:0/734970805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291800e070 con 0x7f292015d470 2023-06-09T04:35:42.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f291effd700 1 -- 172.21.15.92:0/734970805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291800b9c0 con 0x7f292015d470 2023-06-09T04:35:42.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f291effd700 1 -- 172.21.15.92:0/734970805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291800bd70 con 0x7f292015d470 2023-06-09T04:35:42.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 -- 172.21.15.92:0/734970805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f292015d470 msgr2=0x7f292015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 --2- 172.21.15.92:0/734970805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f292015d470 0x7f292015d8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f292006fee0 tx=0x7f291800b330 comp rx=0 tx=0).stop 2023-06-09T04:35:42.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 -- 172.21.15.92:0/734970805 shutdown_connections 2023-06-09T04:35:42.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 --2- 172.21.15.92:0/734970805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f292015d470 0x7f292015d8f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 --2- 172.21.15.92:0/734970805 >> v2:172.21.15.123:3300/0 conn(0x7f29200ce3d0 0x7f29200713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 --2- 172.21.15.92:0/734970805 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29200aed50 0x7f29200ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 -- 172.21.15.92:0/734970805 >> 172.21.15.92:0/734970805 conn(0x7f29200cc380 msgr2=0x7f29200cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 -- 172.21.15.92:0/734970805 shutdown_connections 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.027+0000 7f292683a700 1 -- 172.21.15.92:0/734970805 wait complete. 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.029+0000 7f292683a700 1 Processor -- start 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.029+0000 7f292683a700 1 -- start start 2023-06-09T04:35:42.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f292683a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200aed50 0x7f292015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f292683a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29200ce3d0 0x7f2920152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f292683a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f292015d470 0x7f29200c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f292683a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920153170 con 0x7f29200aed50 2023-06-09T04:35:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f292683a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200c7830 con 0x7f29200ce3d0 2023-06-09T04:35:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f292683a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200c7b10 con 0x7f292015d470 2023-06-09T04:35:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200aed50 0x7f292015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200aed50 0x7f292015bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38392/0 (socket says 172.21.15.92:38392) 2023-06-09T04:35:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 -- 172.21.15.92:0/344376108 learned_addr learned my addr 172.21.15.92:0/344376108 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f2924dd7700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f292015d470 0x7f29200c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291f7fe700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29200ce3d0 0x7f2920152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 -- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f292015d470 msgr2=0x7f29200c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f292015d470 0x7f29200c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 -- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29200ce3d0 msgr2=0x7f2920152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29200ce3d0 0x7f2920152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291ffff700 1 -- 172.21.15.92:0/344376108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918009b30 con 0x7f29200aed50 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.030+0000 7f291f7fe700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29200ce3d0 0x7f2920152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.031+0000 7f291ffff700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200aed50 0x7f292015bd40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f291400e810 tx=0x7f291400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.031+0000 7f291cff9700 1 -- 172.21.15.92:0/344376108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914013610 con 0x7f29200aed50 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.031+0000 7f291cff9700 1 -- 172.21.15.92:0/344376108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2914013770 con 0x7f29200aed50 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.031+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292015b540 con 0x7f29200aed50 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.031+0000 7f291cff9700 1 -- 172.21.15.92:0/344376108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914013b20 con 0x7f29200aed50 2023-06-09T04:35:42.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.031+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29200c61f0 con 0x7f29200aed50 2023-06-09T04:35:42.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.032+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29200c7d30 con 0x7f29200aed50 2023-06-09T04:35:42.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.032+0000 7f291cff9700 1 -- 172.21.15.92:0/344376108 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2914013c80 con 0x7f29200aed50 2023-06-09T04:35:42.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.033+0000 7f291cff9700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f290c06ace0 0x7f290c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.033+0000 7f291cff9700 1 -- 172.21.15.92:0/344376108 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2914089110 con 0x7f29200aed50 2023-06-09T04:35:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.033+0000 7f291f7fe700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f290c06ace0 0x7f290c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.033+0000 7f291f7fe700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f290c06ace0 0x7f290c06d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f29200c57c0 tx=0x7f2910008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.037+0000 7f291cff9700 1 -- 172.21.15.92:0/344376108 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2914057bc0 con 0x7f29200aed50 2023-06-09T04:35:42.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.226+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f292015b370 con 0x7f29200aed50 2023-06-09T04:35:42.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.226+0000 7f291cff9700 1 -- 172.21.15.92:0/344376108 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2914057750 con 0x7f29200aed50 2023-06-09T04:35:42.228 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:42.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f290c06ace0 msgr2=0x7f290c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f290c06ace0 0x7f290c06d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f29200c57c0 tx=0x7f2910008040 comp rx=0 tx=0).stop 2023-06-09T04:35:42.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200aed50 msgr2=0x7f292015bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200aed50 0x7f292015bd40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f291400e810 tx=0x7f291400eb20 comp rx=0 tx=0).stop 2023-06-09T04:35:42.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 shutdown_connections 2023-06-09T04:35:42.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f290c06ace0 0x7f290c06d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f292015d470 0x7f29200c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29200aed50 0x7f292015bd40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 --2- 172.21.15.92:0/344376108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29200ce3d0 0x7f2920152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 >> 172.21.15.92:0/344376108 conn(0x7f29200cc380 msgr2=0x7f292006ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:42.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 shutdown_connections 2023-06-09T04:35:42.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.228+0000 7f292683a700 1 -- 172.21.15.92:0/344376108 wait complete. 2023-06-09T04:35:42.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.363+0000 7f572ef6e700 1 Processor -- start 2023-06-09T04:35:42.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.363+0000 7f572ef6e700 1 -- start start 2023-06-09T04:35:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f572ef6e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f57280b2f90 0x7f57280b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f572ef6e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572815d270 con 0x7f57281569f0 2023-06-09T04:35:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f572ef6e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572815d550 con 0x7f57280b2930 2023-06-09T04:35:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f572ef6e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57280b2db0 con 0x7f57280b2f90 2023-06-09T04:35:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f572cd0a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f57281569f0 0x7f572815c9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36730/0 (socket says 172.21.15.92:36730) 2023-06-09T04:35:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f572cd0a700 1 -- 172.21.15.92:0/2836030827 learned_addr learned my addr 172.21.15.92:0/2836030827 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f5727fff700 1 --2- 172.21.15.92:0/2836030827 >> v2:172.21.15.123:3300/0 conn(0x7f57280b2f90 0x7f57280b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475343627 0 0) 0x7f572815d270 con 0x7f57281569f0 2023-06-09T04:35:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57180035b0 con 0x7f57281569f0 2023-06-09T04:35:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2518730708 0 0) 0x7f57180035b0 con 0x7f57281569f0 2023-06-09T04:35:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 >> v2:172.21.15.123:3300/0 conn(0x7f57280b2f90 msgr2=0x7f57280b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 --2- 172.21.15.92:0/2836030827 >> v2:172.21.15.123:3300/0 conn(0x7f57280b2f90 0x7f57280b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 >> v1:172.21.15.123:6789/0 conn(0x7f57280b2930 legacy=0x7f57280ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572815d830 con 0x7f57281569f0 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f571c001670 con 0x7f57281569f0 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 >> v1:172.21.15.92:6789/0 conn(0x7f57281569f0 legacy=0x7f572815c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718003a00 0x7f5718003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5718004850 0x7f5718004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718005200 0x7f571800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57280b2db0 con 0x7f5718005200 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571800c080 con 0x7f5718004850 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.364+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571800c240 con 0x7f5718003a00 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718005200 0x7f571800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572cd0a700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718003a00 0x7f5718003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f5727fff700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5718004850 0x7f5718004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 -- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718003a00 msgr2=0x7f5718003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718003a00 0x7f5718003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 -- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5718004850 msgr2=0x7f5718004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5718004850 0x7f5718004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 -- 172.21.15.92:0/2836030827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57180035b0 con 0x7f5718005200 2023-06-09T04:35:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f5727fff700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5718004850 0x7f5718004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 -- 172.21.15.92:0/2836030827 reap_dead start 2023-06-09T04:35:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f572d50b700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718005200 0x7f571800ba10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f572000c190 tx=0x7f5718005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810421d468839329 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720001db0 con 0x7f5718005200 2023-06-09T04:35:42.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572000ccf0 con 0x7f5718005200 2023-06-09T04:35:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.365+0000 7f57277fe700 1 -- 172.21.15.92:0/2836030827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720003c40 con 0x7f5718005200 2023-06-09T04:35:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.366+0000 7f572ef6e700 1 -- 172.21.15.92:0/2836030827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718005200 msgr2=0x7f571800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.366+0000 7f572ef6e700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718005200 0x7f571800ba10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f572000c190 tx=0x7f5718005880 comp rx=0 tx=0).stop 2023-06-09T04:35:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.366+0000 7f572ef6e700 1 -- 172.21.15.92:0/2836030827 shutdown_connections 2023-06-09T04:35:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.366+0000 7f572ef6e700 1 --2- 172.21.15.92:0/2836030827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5718005200 0x7f571800ba10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.366+0000 7f572ef6e700 1 -- 172.21.15.92:0/2836030827 >> 172.21.15.92:0/2836030827 conn(0x7f572806e4d0 msgr2=0x7f572806e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:42.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.366+0000 7f572ef6e700 1 -- 172.21.15.92:0/2836030827 shutdown_connections 2023-06-09T04:35:42.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.366+0000 7f572ef6e700 1 -- 172.21.15.92:0/2836030827 wait complete. 2023-06-09T04:35:42.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.367+0000 7f572ef6e700 1 Processor -- start 2023-06-09T04:35:42.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.367+0000 7f572ef6e700 1 -- start start 2023-06-09T04:35:42.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718005200 0x7f57280b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5728157130 0x7f57281575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57280c5be0 0x7f57280c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728157ae0 con 0x7f5728157130 2023-06-09T04:35:42.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57280b6730 con 0x7f57280c5be0 2023-06-09T04:35:42.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57280c6930 con 0x7f5718005200 2023-06-09T04:35:42.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5728157130 0x7f57281575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5728157130 0x7f57281575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38412/0 (socket says 172.21.15.92:38412) 2023-06-09T04:35:42.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 -- 172.21.15.92:0/3748516382 learned_addr learned my addr 172.21.15.92:0/3748516382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:42.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 -- 172.21.15.92:0/3748516382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718005200 msgr2=0x7f57280b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718005200 0x7f57280b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 -- 172.21.15.92:0/3748516382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57280c5be0 msgr2=0x7f57280c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572d50b700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57280c5be0 0x7f57280c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57280c5be0 0x7f57280c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 -- 172.21.15.92:0/3748516382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280c6c10 con 0x7f5728157130 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f5727fff700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5728157130 0x7f57281575a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5714010850 tx=0x7f5714010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f57257fa700 1 -- 172.21.15.92:0/3748516382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714010e50 con 0x7f5728157130 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f57257fa700 1 -- 172.21.15.92:0/3748516382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5714004020 con 0x7f5728157130 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57280c6ef0 con 0x7f5728157130 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f57257fa700 1 -- 172.21.15.92:0/3748516382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140043b0 con 0x7f5728157130 2023-06-09T04:35:42.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.368+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57280c65a0 con 0x7f5728157130 2023-06-09T04:35:42.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.369+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57280b6980 con 0x7f5728157130 2023-06-09T04:35:42.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.370+0000 7f57257fa700 1 -- 172.21.15.92:0/3748516382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f571400e8d0 con 0x7f5728157130 2023-06-09T04:35:42.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.370+0000 7f57257fa700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f570c06ace0 0x7f570c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.370+0000 7f57257fa700 1 -- 172.21.15.92:0/3748516382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57140896f0 con 0x7f5728157130 2023-06-09T04:35:42.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.370+0000 7f572cd0a700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f570c06ace0 0x7f570c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.371+0000 7f572cd0a700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f570c06ace0 0x7f570c06d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f571c000f60 tx=0x7f571c000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.375+0000 7f57257fa700 1 -- 172.21.15.92:0/3748516382 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57140582c0 con 0x7f5728157130 2023-06-09T04:35:42.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.559+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57280c7410 con 0x7f5728157130 2023-06-09T04:35:42.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.560+0000 7f57257fa700 1 -- 172.21.15.92:0/3748516382 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5714057e50 con 0x7f5728157130 2023-06-09T04:35:42.561 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.561+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f570c06ace0 msgr2=0x7f570c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.561+0000 7f572ef6e700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f570c06ace0 0x7f570c06d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f571c000f60 tx=0x7f571c000ed0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.561+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5728157130 msgr2=0x7f57281575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.561+0000 7f572ef6e700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5728157130 0x7f57281575a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5714010850 tx=0x7f5714010c10 comp rx=0 tx=0).stop 2023-06-09T04:35:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 shutdown_connections 2023-06-09T04:35:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f570c06ace0 0x7f570c06d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57280c5be0 0x7f57280c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5718005200 0x7f57280b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 --2- 172.21.15.92:0/3748516382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5728157130 0x7f57281575a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 >> 172.21.15.92:0/3748516382 conn(0x7f572806e4d0 msgr2=0x7f572815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 shutdown_connections 2023-06-09T04:35:42.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.562+0000 7f572ef6e700 1 -- 172.21.15.92:0/3748516382 wait complete. 2023-06-09T04:35:42.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:42.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.693+0000 7fead9daa700 1 Processor -- start 2023-06-09T04:35:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.693+0000 7fead9daa700 1 -- start start 2023-06-09T04:35:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead9daa700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fead415c7d0 0x7fead40ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead9daa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fead40cf010 0x7fead406e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead9daa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40ce730 con 0x7fead40ac400 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead9daa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead406aaf0 con 0x7fead40cf010 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead9daa700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead415cba0 con 0x7fead415c7d0 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fead40cf010 0x7fead406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fead40cf010 0x7fead406e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38422/0 (socket says 172.21.15.92:38422) 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead3fff700 1 -- 172.21.15.92:0/1143019618 learned_addr learned my addr 172.21.15.92:0/1143019618 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead37fe700 1 --2- 172.21.15.92:0/1143019618 >> v2:172.21.15.123:3300/0 conn(0x7fead415c7d0 0x7fead40ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead3fff700 1 -- 172.21.15.92:0/1143019618 >> v2:172.21.15.123:3300/0 conn(0x7fead415c7d0 msgr2=0x7fead40ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:42.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead3fff700 1 --2- 172.21.15.92:0/1143019618 >> v2:172.21.15.123:3300/0 conn(0x7fead415c7d0 0x7fead40ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead3fff700 1 -- 172.21.15.92:0/1143019618 >> v1:172.21.15.123:6789/0 conn(0x7fead40ac400 legacy=0x7fead40ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:42.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.694+0000 7fead3fff700 1 -- 172.21.15.92:0/1143019618 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead4069ee0 con 0x7fead40cf010 2023-06-09T04:35:42.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead3fff700 1 --2- 172.21.15.92:0/1143019618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead40cf010 0x7fead406e220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fead4071870 tx=0x7feacc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13997432b0d1c3da server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead27fc700 1 -- 172.21.15.92:0/1143019618 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc00e070 con 0x7fead40cf010 2023-06-09T04:35:42.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead27fc700 1 -- 172.21.15.92:0/1143019618 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc003a20 con 0x7fead40cf010 2023-06-09T04:35:42.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead27fc700 1 -- 172.21.15.92:0/1143019618 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc003dd0 con 0x7fead40cf010 2023-06-09T04:35:42.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 -- 172.21.15.92:0/1143019618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead40cf010 msgr2=0x7fead406e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 --2- 172.21.15.92:0/1143019618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead40cf010 0x7fead406e220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fead4071870 tx=0x7feacc00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 -- 172.21.15.92:0/1143019618 shutdown_connections 2023-06-09T04:35:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 --2- 172.21.15.92:0/1143019618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead40cf010 0x7fead406e220 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 --2- 172.21.15.92:0/1143019618 >> v2:172.21.15.123:3300/0 conn(0x7fead415c7d0 0x7fead40ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 -- 172.21.15.92:0/1143019618 >> 172.21.15.92:0/1143019618 conn(0x7fead4098c10 msgr2=0x7fead40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 -- 172.21.15.92:0/1143019618 shutdown_connections 2023-06-09T04:35:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.695+0000 7fead9daa700 1 -- 172.21.15.92:0/1143019618 wait complete. 2023-06-09T04:35:42.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.697+0000 7fead9daa700 1 Processor -- start 2023-06-09T04:35:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.697+0000 7fead9daa700 1 -- start start 2023-06-09T04:35:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.697+0000 7fead9daa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead40ac400 0x7fead415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.697+0000 7fead9daa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fead40cf010 0x7fead40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.697+0000 7fead9daa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead415c7d0 0x7fead40c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead9daa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40cab40 con 0x7fead415c7d0 2023-06-09T04:35:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead9daa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40c6160 con 0x7fead40ac400 2023-06-09T04:35:42.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead9daa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40c5e80 con 0x7fead40cf010 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead415c7d0 0x7fead40c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead415c7d0 0x7fead40c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38428/0 (socket says 172.21.15.92:38428) 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 -- 172.21.15.92:0/4280299022 learned_addr learned my addr 172.21.15.92:0/4280299022 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead37fe700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead40ac400 0x7fead415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead2ffd700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fead40cf010 0x7fead40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 -- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fead40cf010 msgr2=0x7fead40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fead40cf010 0x7fead40c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 -- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead40ac400 msgr2=0x7fead415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead40ac400 0x7fead415b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 -- 172.21.15.92:0/4280299022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feacc009b30 con 0x7fead415c7d0 2023-06-09T04:35:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead3fff700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead415c7d0 0x7fead40c5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feacc015040 tx=0x7feacc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7feabbfff700 1 -- 172.21.15.92:0/4280299022 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc004240 con 0x7fead415c7d0 2023-06-09T04:35:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7feabbfff700 1 -- 172.21.15.92:0/4280299022 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc0043c0 con 0x7fead415c7d0 2023-06-09T04:35:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead40ca4f0 con 0x7fead415c7d0 2023-06-09T04:35:42.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7feabbfff700 1 -- 172.21.15.92:0/4280299022 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc004770 con 0x7fead415c7d0 2023-06-09T04:35:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.698+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead414e380 con 0x7fead415c7d0 2023-06-09T04:35:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.699+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead414e510 con 0x7fead415c7d0 2023-06-09T04:35:42.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.700+0000 7feabbfff700 1 -- 172.21.15.92:0/4280299022 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feacc019070 con 0x7fead415c7d0 2023-06-09T04:35:42.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.700+0000 7feabbfff700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7feac006ace0 0x7feac006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:42.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.700+0000 7feabbfff700 1 -- 172.21.15.92:0/4280299022 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feacc08a330 con 0x7fead415c7d0 2023-06-09T04:35:42.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.700+0000 7fead37fe700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7feac006ace0 0x7feac006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:42.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.701+0000 7fead37fe700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7feac006ace0 0x7feac006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feac8006fd0 tx=0x7feac8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:42.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.704+0000 7feabbfff700 1 -- 172.21.15.92:0/4280299022 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feacc055470 con 0x7fead415c7d0 2023-06-09T04:35:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.893+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fead40ce730 con 0x7fead415c7d0 2023-06-09T04:35:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.894+0000 7feabbfff700 1 -- 172.21.15.92:0/4280299022 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7fead40ce730 con 0x7fead415c7d0 2023-06-09T04:35:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:42.896 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:42.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7feac006ace0 msgr2=0x7feac006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7feac006ace0 0x7feac006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feac8006fd0 tx=0x7feac8008040 comp rx=0 tx=0).stop 2023-06-09T04:35:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead415c7d0 msgr2=0x7fead40c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead415c7d0 0x7fead40c5870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feacc015040 tx=0x7feacc004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 shutdown_connections 2023-06-09T04:35:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7feac006ace0 0x7feac006d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fead415c7d0 0x7fead40c5870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fead40ac400 0x7fead415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 --2- 172.21.15.92:0/4280299022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fead40cf010 0x7fead40c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 >> 172.21.15.92:0/4280299022 conn(0x7fead4098c10 msgr2=0x7fead406e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 shutdown_connections 2023-06-09T04:35:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:42.896+0000 7fead9daa700 1 -- 172.21.15.92:0/4280299022 wait complete. 2023-06-09T04:35:42.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:42.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:43.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.031+0000 7f467901b700 1 Processor -- start 2023-06-09T04:35:43.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.031+0000 7f467901b700 1 -- start start 2023-06-09T04:35:43.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467901b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4674069db0 0x7f467415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467901b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f46740cce60 0x7f467406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467901b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467415d040 con 0x7f46740cf440 2023-06-09T04:35:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467901b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4674160ab0 con 0x7f4674069db0 2023-06-09T04:35:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467901b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467415ca10 con 0x7f46740cce60 2023-06-09T04:35:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4672d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4674069db0 0x7f467415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f46740cf440 0x7f467406a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36766/0 (socket says 172.21.15.92:36766) 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467259c700 1 -- 172.21.15.92:0/2124534206 learned_addr learned my addr 172.21.15.92:0/2124534206 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f467359e700 1 --2- 172.21.15.92:0/2124534206 >> v2:172.21.15.123:3300/0 conn(0x7f46740cce60 0x7f467406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4672d9d700 1 -- 172.21.15.92:0/2124534206 >> v2:172.21.15.123:3300/0 conn(0x7f46740cce60 msgr2=0x7f467406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4672d9d700 1 --2- 172.21.15.92:0/2124534206 >> v2:172.21.15.123:3300/0 conn(0x7f46740cce60 0x7f467406ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4672d9d700 1 -- 172.21.15.92:0/2124534206 >> v1:172.21.15.92:6789/0 conn(0x7f46740cf440 legacy=0x7f467406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4671d9b700 1 -- 172.21.15.92:0/2124534206 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172936985 0 0) 0x7f467415d040 con 0x7f46740cf440 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4672d9d700 1 -- 172.21.15.92:0/2124534206 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674160d50 con 0x7f4674069db0 2023-06-09T04:35:43.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4672d9d700 1 --2- 172.21.15.92:0/2124534206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4674069db0 0x7f467415d4d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4674071340 tx=0x7f4664009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607a9c74218240bd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4671d9b700 1 -- 172.21.15.92:0/2124534206 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664011830 con 0x7f4674069db0 2023-06-09T04:35:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4671d9b700 1 -- 172.21.15.92:0/2124534206 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4664011990 con 0x7f4674069db0 2023-06-09T04:35:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.032+0000 7f4671d9b700 1 -- 172.21.15.92:0/2124534206 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4664011d20 con 0x7f4674069db0 2023-06-09T04:35:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 -- 172.21.15.92:0/2124534206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4674069db0 msgr2=0x7f467415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 --2- 172.21.15.92:0/2124534206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4674069db0 0x7f467415d4d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4674071340 tx=0x7f4664009b20 comp rx=0 tx=0).stop 2023-06-09T04:35:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 -- 172.21.15.92:0/2124534206 shutdown_connections 2023-06-09T04:35:43.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 --2- 172.21.15.92:0/2124534206 >> v2:172.21.15.123:3300/0 conn(0x7f46740cce60 0x7f467406ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 --2- 172.21.15.92:0/2124534206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4674069db0 0x7f467415d4d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 -- 172.21.15.92:0/2124534206 >> 172.21.15.92:0/2124534206 conn(0x7f46740cc380 msgr2=0x7f46740cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:43.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 -- 172.21.15.92:0/2124534206 shutdown_connections 2023-06-09T04:35:43.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.033+0000 7f467901b700 1 -- 172.21.15.92:0/2124534206 wait complete. 2023-06-09T04:35:43.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.034+0000 7f467901b700 1 Processor -- start 2023-06-09T04:35:43.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.034+0000 7f467901b700 1 -- start start 2023-06-09T04:35:43.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467901b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4674069db0 0x7f46740b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467901b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46740cce60 0x7f4674156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467901b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46740cf440 0x7f4674154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467901b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46740b04b0 con 0x7f46740cce60 2023-06-09T04:35:43.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467901b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46741592e0 con 0x7f46740cf440 2023-06-09T04:35:43.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467901b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46740af9b0 con 0x7f4674069db0 2023-06-09T04:35:43.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46740cce60 0x7f4674156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46740cce60 0x7f4674156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38440/0 (socket says 172.21.15.92:38440) 2023-06-09T04:35:43.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467259c700 1 -- 172.21.15.92:0/1514764447 learned_addr learned my addr 172.21.15.92:0/1514764447 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:43.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f467359e700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46740cf440 0x7f4674154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.035+0000 7f4672d9d700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4674069db0 0x7f46740b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467259c700 1 -- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4674069db0 msgr2=0x7f46740b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467259c700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4674069db0 0x7f46740b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467259c700 1 -- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46740cf440 msgr2=0x7f4674154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467259c700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46740cf440 0x7f4674154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467259c700 1 -- 172.21.15.92:0/1514764447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46640094a0 con 0x7f46740cce60 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467259c700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46740cce60 0x7f4674156370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f466800c550 tx=0x7f466800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f465b7fe700 1 -- 172.21.15.92:0/1514764447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668004c80 con 0x7f46740cce60 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f465b7fe700 1 -- 172.21.15.92:0/1514764447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668004e00 con 0x7f46740cce60 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4674158a80 con 0x7f46740cce60 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f465b7fe700 1 -- 172.21.15.92:0/1514764447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668007310 con 0x7f46740cce60 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.036+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4674159000 con 0x7f46740cce60 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.037+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467406be50 con 0x7f46740cce60 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.037+0000 7f465b7fe700 1 -- 172.21.15.92:0/1514764447 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4668007470 con 0x7f46740cce60 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.038+0000 7f465b7fe700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f466006ace0 0x7f466006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.038+0000 7f465b7fe700 1 -- 172.21.15.92:0/1514764447 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4668089240 con 0x7f46740cce60 2023-06-09T04:35:43.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.038+0000 7f4672d9d700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f466006ace0 0x7f466006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.038+0000 7f4672d9d700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f466006ace0 0x7f466006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46640090b0 tx=0x7f4664008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.042+0000 7f465b7fe700 1 -- 172.21.15.92:0/1514764447 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f46680542f0 con 0x7f46740cce60 2023-06-09T04:35:43.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.231+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f467404e8e0 con 0x7f46740cce60 2023-06-09T04:35:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.231+0000 7f465b7fe700 1 -- 172.21.15.92:0/1514764447 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4668057910 con 0x7f46740cce60 2023-06-09T04:35:43.233 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f466006ace0 msgr2=0x7f466006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f466006ace0 0x7f466006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f46640090b0 tx=0x7f4664008030 comp rx=0 tx=0).stop 2023-06-09T04:35:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46740cce60 msgr2=0x7f4674156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46740cce60 0x7f4674156370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f466800c550 tx=0x7f466800c910 comp rx=0 tx=0).stop 2023-06-09T04:35:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 shutdown_connections 2023-06-09T04:35:43.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f466006ace0 0x7f466006d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46740cf440 0x7f4674154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4674069db0 0x7f46740b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 --2- 172.21.15.92:0/1514764447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46740cce60 0x7f4674156370 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 >> 172.21.15.92:0/1514764447 conn(0x7f46740cc380 msgr2=0x7f467406f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:43.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 shutdown_connections 2023-06-09T04:35:43.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.233+0000 7f467901b700 1 -- 172.21.15.92:0/1514764447 wait complete. 2023-06-09T04:35:43.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:43.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.370+0000 7f079ee6b700 1 Processor -- start 2023-06-09T04:35:43.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.370+0000 7f079ee6b700 1 -- start start 2023-06-09T04:35:43.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.370+0000 7f079ee6b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f07980cf440 0x7f079815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.370+0000 7f079ee6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cce60 0x7f079806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.370+0000 7f079ee6b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079815d040 con 0x7f0798069db0 2023-06-09T04:35:43.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.370+0000 7f079ee6b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0798160ab0 con 0x7f07980cf440 2023-06-09T04:35:43.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.370+0000 7f079ee6b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079815ca10 con 0x7f07980cce60 2023-06-09T04:35:43.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f0797fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f07980cf440 0x7f079815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079d408700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cce60 0x7f079806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079cc07700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0798069db0 0x7f079806a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59274/0 (socket says 172.21.15.92:59274) 2023-06-09T04:35:43.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079cc07700 1 -- 172.21.15.92:0/1325232648 learned_addr learned my addr 172.21.15.92:0/1325232648 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:43.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079d408700 1 -- 172.21.15.92:0/1325232648 >> v2:172.21.15.123:3300/0 conn(0x7f07980cf440 msgr2=0x7f079815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:43.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079d408700 1 --2- 172.21.15.92:0/1325232648 >> v2:172.21.15.123:3300/0 conn(0x7f07980cf440 0x7f079815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079d408700 1 -- 172.21.15.92:0/1325232648 >> v1:172.21.15.123:6789/0 conn(0x7f0798069db0 legacy=0x7f079806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079d408700 1 -- 172.21.15.92:0/1325232648 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798160d50 con 0x7f07980cce60 2023-06-09T04:35:43.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f079d408700 1 --2- 172.21.15.92:0/1325232648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cce60 0x7f079806ee20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0790009d90 tx=0x7f079000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10919ac74c7c636 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.371+0000 7f07977fe700 1 -- 172.21.15.92:0/1325232648 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000e070 con 0x7f07980cce60 2023-06-09T04:35:43.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f07977fe700 1 -- 172.21.15.92:0/1325232648 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000bb20 con 0x7f07980cce60 2023-06-09T04:35:43.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f07977fe700 1 -- 172.21.15.92:0/1325232648 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790012430 con 0x7f07980cce60 2023-06-09T04:35:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 -- 172.21.15.92:0/1325232648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cce60 msgr2=0x7f079806ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 --2- 172.21.15.92:0/1325232648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cce60 0x7f079806ee20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0790009d90 tx=0x7f079000b330 comp rx=0 tx=0).stop 2023-06-09T04:35:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 -- 172.21.15.92:0/1325232648 shutdown_connections 2023-06-09T04:35:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 --2- 172.21.15.92:0/1325232648 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cce60 0x7f079806ee20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 --2- 172.21.15.92:0/1325232648 >> v2:172.21.15.123:3300/0 conn(0x7f07980cf440 0x7f079815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 -- 172.21.15.92:0/1325232648 >> 172.21.15.92:0/1325232648 conn(0x7f07980cc380 msgr2=0x7f07980cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:43.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 -- 172.21.15.92:0/1325232648 shutdown_connections 2023-06-09T04:35:43.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.372+0000 7f079ee6b700 1 -- 172.21.15.92:0/1325232648 wait complete. 2023-06-09T04:35:43.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.373+0000 7f079ee6b700 1 Processor -- start 2023-06-09T04:35:43.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.373+0000 7f079ee6b700 1 -- start start 2023-06-09T04:35:43.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079ee6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798069db0 0x7f07980c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079ee6b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07980cce60 0x7f079815f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079ee6b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cf440 0x7f0798154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079ee6b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07980b2730 con 0x7f0798069db0 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079ee6b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07980b2a10 con 0x7f07980cce60 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079ee6b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07980b1ad0 con 0x7f07980cf440 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079cc07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798069db0 0x7f07980c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079cc07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798069db0 0x7f07980c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38452/0 (socket says 172.21.15.92:38452) 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.374+0000 7f079cc07700 1 -- 172.21.15.92:0/3731981297 learned_addr learned my addr 172.21.15.92:0/3731981297 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:43.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f0797fff700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07980cce60 0x7f079815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079cc07700 1 -- 172.21.15.92:0/3731981297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cf440 msgr2=0x7f0798154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079cc07700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cf440 0x7f0798154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079cc07700 1 -- 172.21.15.92:0/3731981297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07980cce60 msgr2=0x7f079815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079cc07700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07980cce60 0x7f079815f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079cc07700 1 -- 172.21.15.92:0/3731981297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790009a70 con 0x7f0798069db0 2023-06-09T04:35:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079cc07700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798069db0 0x7f07980c9750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c550 tx=0x7f078c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f07957fa700 1 -- 172.21.15.92:0/3731981297 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c005d70 con 0x7f0798069db0 2023-06-09T04:35:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f07957fa700 1 -- 172.21.15.92:0/3731981297 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c0070d0 con 0x7f0798069db0 2023-06-09T04:35:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07980b1db0 con 0x7f0798069db0 2023-06-09T04:35:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f07957fa700 1 -- 172.21.15.92:0/3731981297 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c007440 con 0x7f0798069db0 2023-06-09T04:35:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.375+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0798155f40 con 0x7f0798069db0 2023-06-09T04:35:43.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.376+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079806be50 con 0x7f0798069db0 2023-06-09T04:35:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.376+0000 7f07957fa700 1 -- 172.21.15.92:0/3731981297 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f078c0075a0 con 0x7f0798069db0 2023-06-09T04:35:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.376+0000 7f07957fa700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f077c06ace0 0x7f077c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.377+0000 7f07957fa700 1 -- 172.21.15.92:0/3731981297 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f078c0888b0 con 0x7f0798069db0 2023-06-09T04:35:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.377+0000 7f0797fff700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f077c06ace0 0x7f077c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.377+0000 7f0797fff700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f077c06ace0 0x7f077c06d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07980b56b0 tx=0x7f0784008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.381+0000 7f07957fa700 1 -- 172.21.15.92:0/3731981297 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f078c053a10 con 0x7f0798069db0 2023-06-09T04:35:43.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.569+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f079804e8e0 con 0x7f0798069db0 2023-06-09T04:35:43.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.569+0000 7f07957fa700 1 -- 172.21.15.92:0/3731981297 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f078c057030 con 0x7f0798069db0 2023-06-09T04:35:43.571 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:43.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.571+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f077c06ace0 msgr2=0x7f077c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.571+0000 7f079ee6b700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f077c06ace0 0x7f077c06d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07980b56b0 tx=0x7f0784008040 comp rx=0 tx=0).stop 2023-06-09T04:35:43.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.571+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798069db0 msgr2=0x7f07980c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.571+0000 7f079ee6b700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798069db0 0x7f07980c9750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f078c00c550 tx=0x7f078c00c910 comp rx=0 tx=0).stop 2023-06-09T04:35:43.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 shutdown_connections 2023-06-09T04:35:43.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f077c06ace0 0x7f077c06d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07980cf440 0x7f0798154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0798069db0 0x7f07980c9750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 --2- 172.21.15.92:0/3731981297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07980cce60 0x7f079815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 >> 172.21.15.92:0/3731981297 conn(0x7f07980cc380 msgr2=0x7f07980cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:43.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 shutdown_connections 2023-06-09T04:35:43.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.572+0000 7f079ee6b700 1 -- 172.21.15.92:0/3731981297 wait complete. 2023-06-09T04:35:43.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:43.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.707+0000 7f994593b700 1 Processor -- start 2023-06-09T04:35:43.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.707+0000 7f994593b700 1 -- start start 2023-06-09T04:35:43.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f994593b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f99400cf100 0x7f99400d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f994593b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99400d2b00 con 0x7f99400d3800 2023-06-09T04:35:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f994593b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99400cd3c0 con 0x7f994015ced0 2023-06-09T04:35:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f994593b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400cd6a0 con 0x7f99400cf100 2023-06-09T04:35:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f993f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f99400d3800 0x7f99400cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36796/0 (socket says 172.21.15.92:36796) 2023-06-09T04:35:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f993f7fe700 1 -- 172.21.15.92:0/4209576594 learned_addr learned my addr 172.21.15.92:0/4209576594 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309314741 0 0) 0x7f99400d2b00 con 0x7f99400d3800 2023-06-09T04:35:43.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f993e7fc700 1 --2- 172.21.15.92:0/4209576594 >> v2:172.21.15.123:3300/0 conn(0x7f99400cf100 0x7f99400d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.708+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99280035b0 con 0x7f99400d3800 2023-06-09T04:35:43.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2895850606 0 0) 0x7f99280035b0 con 0x7f99400d3800 2023-06-09T04:35:43.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 >> v2:172.21.15.123:3300/0 conn(0x7f99400cf100 msgr2=0x7f99400d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:43.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 --2- 172.21.15.92:0/4209576594 >> v2:172.21.15.123:3300/0 conn(0x7f99400cf100 0x7f99400d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 >> v1:172.21.15.123:6789/0 conn(0x7f994015ced0 legacy=0x7f994015d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400d2da0 con 0x7f99400d3800 2023-06-09T04:35:43.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f99380019f0 con 0x7f99400d3800 2023-06-09T04:35:43.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 >> v1:172.21.15.92:6789/0 conn(0x7f99400d3800 legacy=0x7f99400cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9928003a00 0x7f9928003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9928004850 0x7f9928004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f992800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400cd6a0 con 0x7f9928005200 2023-06-09T04:35:43.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992800c080 con 0x7f9928003a00 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992800c240 con 0x7f9928004850 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f992800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993effd700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9928003a00 0x7f9928003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993e7fc700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9928004850 0x7f9928004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 -- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9928004850 msgr2=0x7f9928004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9928004850 0x7f9928004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 -- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9928003a00 msgr2=0x7f9928003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:43.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9928003a00 0x7f9928003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 -- 172.21.15.92:0/4209576594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99280035b0 con 0x7f9928005200 2023-06-09T04:35:43.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 -- 172.21.15.92:0/4209576594 reap_dead start 2023-06-09T04:35:43.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993f7fe700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f992800ba10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f993800a830 tx=0x7f9928005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20311f34f12b86fe server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993800a2f0 con 0x7f9928005200 2023-06-09T04:35:43.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9938012600 con 0x7f9928005200 2023-06-09T04:35:43.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.709+0000 7f993dffb700 1 -- 172.21.15.92:0/4209576594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380129b0 con 0x7f9928005200 2023-06-09T04:35:43.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.710+0000 7f994593b700 1 -- 172.21.15.92:0/4209576594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 msgr2=0x7f992800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.710+0000 7f994593b700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f992800ba10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f993800a830 tx=0x7f9928005880 comp rx=0 tx=0).stop 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.710+0000 7f994593b700 1 -- 172.21.15.92:0/4209576594 shutdown_connections 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.710+0000 7f994593b700 1 --2- 172.21.15.92:0/4209576594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f992800ba10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.710+0000 7f994593b700 1 -- 172.21.15.92:0/4209576594 >> 172.21.15.92:0/4209576594 conn(0x7f994006b2e0 msgr2=0x7f994006b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.710+0000 7f994593b700 1 -- 172.21.15.92:0/4209576594 shutdown_connections 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.710+0000 7f994593b700 1 -- 172.21.15.92:0/4209576594 wait complete. 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.712+0000 7f994593b700 1 Processor -- start 2023-06-09T04:35:43.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.712+0000 7f994593b700 1 -- start start 2023-06-09T04:35:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f99400b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99400cca30 0x7f9940151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99400b7d90 0x7f99400b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400b8c60 con 0x7f9928005200 2023-06-09T04:35:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9940151fb0 con 0x7f99400cca30 2023-06-09T04:35:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99401f2070 con 0x7f99400b7d90 2023-06-09T04:35:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f99400b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f99400b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38472/0 (socket says 172.21.15.92:38472) 2023-06-09T04:35:43.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 -- 172.21.15.92:0/1222343404 learned_addr learned my addr 172.21.15.92:0/1222343404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993f7fe700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99400b7d90 0x7f99400b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993e7fc700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99400cca30 0x7f9940151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 -- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99400b7d90 msgr2=0x7f99400b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99400b7d90 0x7f99400b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 -- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99400cca30 msgr2=0x7f9940151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99400cca30 0x7f9940151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 -- 172.21.15.92:0/1222343404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99401f2350 con 0x7f9928005200 2023-06-09T04:35:43.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993f7fe700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99400b7d90 0x7f99400b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f993effd700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f99400b64a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f993400f820 tx=0x7f993400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f9927fff700 1 -- 172.21.15.92:0/1222343404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340052e0 con 0x7f9928005200 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f9927fff700 1 -- 172.21.15.92:0/1222343404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9934005460 con 0x7f9928005200 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401f2630 con 0x7f9928005200 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f9927fff700 1 -- 172.21.15.92:0/1222343404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340038c0 con 0x7f9928005200 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.713+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401f2bf0 con 0x7f9928005200 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.714+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99401f2d80 con 0x7f9928005200 2023-06-09T04:35:43.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.715+0000 7f9927fff700 1 -- 172.21.15.92:0/1222343404 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9934003a20 con 0x7f9928005200 2023-06-09T04:35:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.715+0000 7f9927fff700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f992c06ace0 0x7f992c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.715+0000 7f9927fff700 1 -- 172.21.15.92:0/1222343404 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f99340891e0 con 0x7f9928005200 2023-06-09T04:35:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.715+0000 7f993e7fc700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f992c06ace0 0x7f992c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.715+0000 7f993e7fc700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f992c06ace0 0x7f992c06d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99400d27e0 tx=0x7f9930008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:43.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.719+0000 7f9927fff700 1 -- 172.21.15.92:0/1222343404 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9934054320 con 0x7f9928005200 2023-06-09T04:35:43.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.906+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f994004e8e0 con 0x7f9928005200 2023-06-09T04:35:43.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.907+0000 7f9927fff700 1 -- 172.21.15.92:0/1222343404 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9934057940 con 0x7f9928005200 2023-06-09T04:35:43.909 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:43.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f992c06ace0 msgr2=0x7f992c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f992c06ace0 0x7f992c06d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f99400d27e0 tx=0x7f9930008040 comp rx=0 tx=0).stop 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 msgr2=0x7f99400b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f99400b64a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f993400f820 tx=0x7f993400fbe0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 shutdown_connections 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f992c06ace0 0x7f992c06d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f99400b7d90 0x7f99400b8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9928005200 0x7f99400b64a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 --2- 172.21.15.92:0/1222343404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99400cca30 0x7f9940151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:43.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 >> 172.21.15.92:0/1222343404 conn(0x7f994006b2e0 msgr2=0x7f99400d09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:43.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 shutdown_connections 2023-06-09T04:35:43.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:43.909+0000 7f994593b700 1 -- 172.21.15.92:0/1222343404 wait complete. 2023-06-09T04:35:43.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd471d70700 1 Processor -- start 2023-06-09T04:35:44.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd471d70700 1 -- start start 2023-06-09T04:35:44.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd471d70700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c069db0 0x7fd46c15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd471d70700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd46c0cce60 0x7fd46c06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd471d70700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c15d040 con 0x7fd46c0cf440 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd471d70700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c160ab0 con 0x7fd46c0cce60 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd471d70700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c15ca10 con 0x7fd46c069db0 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd46bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd46c0cce60 0x7fd46c06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd46affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd46c0cf440 0x7fd46c06a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36812/0 (socket says 172.21.15.92:36812) 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.044+0000 7fd46affd700 1 -- 172.21.15.92:0/89675619 learned_addr learned my addr 172.21.15.92:0/89675619 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:44.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46bfff700 1 -- 172.21.15.92:0/89675619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c069db0 msgr2=0x7fd46c15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46b7fe700 1 --2- 172.21.15.92:0/89675619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c069db0 0x7fd46c15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46bfff700 1 --2- 172.21.15.92:0/89675619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c069db0 0x7fd46c15d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46bfff700 1 -- 172.21.15.92:0/89675619 >> v1:172.21.15.92:6789/0 conn(0x7fd46c0cf440 legacy=0x7fd46c06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46bfff700 1 -- 172.21.15.92:0/89675619 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c160d50 con 0x7fd46c0cce60 2023-06-09T04:35:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46a7fc700 1 -- 172.21.15.92:0/89675619 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403117606 0 0) 0x7fd46c15d040 con 0x7fd46c0cf440 2023-06-09T04:35:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46bfff700 1 --2- 172.21.15.92:0/89675619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c0cce60 0x7fd46c06ee20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd464009d90 tx=0x7fd46400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be8169f54ac9412 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46a7fc700 1 -- 172.21.15.92:0/89675619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400e070 con 0x7fd46c0cce60 2023-06-09T04:35:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46a7fc700 1 -- 172.21.15.92:0/89675619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46400bbd0 con 0x7fd46c0cce60 2023-06-09T04:35:44.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.045+0000 7fd46a7fc700 1 -- 172.21.15.92:0/89675619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464012430 con 0x7fd46c0cce60 2023-06-09T04:35:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 -- 172.21.15.92:0/89675619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c0cce60 msgr2=0x7fd46c06ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 --2- 172.21.15.92:0/89675619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c0cce60 0x7fd46c06ee20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd464009d90 tx=0x7fd46400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 -- 172.21.15.92:0/89675619 shutdown_connections 2023-06-09T04:35:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 --2- 172.21.15.92:0/89675619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c069db0 0x7fd46c15d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 --2- 172.21.15.92:0/89675619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c0cce60 0x7fd46c06ee20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 -- 172.21.15.92:0/89675619 >> 172.21.15.92:0/89675619 conn(0x7fd46c0cc380 msgr2=0x7fd46c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:44.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 -- 172.21.15.92:0/89675619 shutdown_connections 2023-06-09T04:35:44.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.046+0000 7fd471d70700 1 -- 172.21.15.92:0/89675619 wait complete. 2023-06-09T04:35:44.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.047+0000 7fd471d70700 1 Processor -- start 2023-06-09T04:35:44.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.047+0000 7fd471d70700 1 -- start start 2023-06-09T04:35:44.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd471d70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c069db0 0x7fd46c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd471d70700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c0cce60 0x7fd46c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd471d70700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c0cf440 0x7fd46c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd471d70700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0b0a20 con 0x7fd46c069db0 2023-06-09T04:35:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd471d70700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0b0d00 con 0x7fd46c0cce60 2023-06-09T04:35:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd471d70700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0b0fe0 con 0x7fd46c0cf440 2023-06-09T04:35:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c069db0 0x7fd46c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c069db0 0x7fd46c155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38484/0 (socket says 172.21.15.92:38484) 2023-06-09T04:35:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 -- 172.21.15.92:0/449913788 learned_addr learned my addr 172.21.15.92:0/449913788 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:44.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46affd700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c0cce60 0x7fd46c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46bfff700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c0cf440 0x7fd46c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 -- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c0cf440 msgr2=0x7fd46c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c0cf440 0x7fd46c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 -- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c0cce60 msgr2=0x7fd46c0ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c0cce60 0x7fd46c0ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 -- 172.21.15.92:0/449913788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd464009a70 con 0x7fd46c069db0 2023-06-09T04:35:44.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46bfff700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c0cf440 0x7fd46c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.048+0000 7fd46b7fe700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c069db0 0x7fd46c155f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c7f0 tx=0x7fd45c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.049+0000 7fd463fff700 1 -- 172.21.15.92:0/449913788 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c011800 con 0x7fd46c069db0 2023-06-09T04:35:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.049+0000 7fd463fff700 1 -- 172.21.15.92:0/449913788 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c011960 con 0x7fd46c069db0 2023-06-09T04:35:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.049+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c0b12c0 con 0x7fd46c069db0 2023-06-09T04:35:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.049+0000 7fd463fff700 1 -- 172.21.15.92:0/449913788 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c011cf0 con 0x7fd46c069db0 2023-06-09T04:35:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.049+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c155be0 con 0x7fd46c069db0 2023-06-09T04:35:44.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.049+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c0b17e0 con 0x7fd46c069db0 2023-06-09T04:35:44.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.050+0000 7fd463fff700 1 -- 172.21.15.92:0/449913788 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd45c018360 con 0x7fd46c069db0 2023-06-09T04:35:44.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.050+0000 7fd463fff700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd44c06ace0 0x7fd44c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.050+0000 7fd463fff700 1 -- 172.21.15.92:0/449913788 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd45c089550 con 0x7fd46c069db0 2023-06-09T04:35:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.050+0000 7fd46affd700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd44c06ace0 0x7fd44c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.051+0000 7fd46affd700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd44c06ace0 0x7fd44c06d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd46c1f5830 tx=0x7fd454007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.055+0000 7fd463fff700 1 -- 172.21.15.92:0/449913788 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd45c058000 con 0x7fd46c069db0 2023-06-09T04:35:44.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.244+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd46c160260 con 0x7fd46c069db0 2023-06-09T04:35:44.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.244+0000 7fd463fff700 1 -- 172.21.15.92:0/449913788 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd45c057b90 con 0x7fd46c069db0 2023-06-09T04:35:44.245 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:44.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd44c06ace0 msgr2=0x7fd44c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd44c06ace0 0x7fd44c06d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd46c1f5830 tx=0x7fd454007c90 comp rx=0 tx=0).stop 2023-06-09T04:35:44.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c069db0 msgr2=0x7fd46c155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c069db0 0x7fd46c155f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c7f0 tx=0x7fd45c00cbb0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 shutdown_connections 2023-06-09T04:35:44.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd44c06ace0 0x7fd44c06d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd46c0cf440 0x7fd46c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd46c069db0 0x7fd46c155f70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 --2- 172.21.15.92:0/449913788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd46c0cce60 0x7fd46c0ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 >> 172.21.15.92:0/449913788 conn(0x7fd46c0cc380 msgr2=0x7fd46c15fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:44.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 shutdown_connections 2023-06-09T04:35:44.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.246+0000 7fd471d70700 1 -- 172.21.15.92:0/449913788 wait complete. 2023-06-09T04:35:44.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:44.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.377+0000 7f07b441d700 1 Processor -- start 2023-06-09T04:35:44.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.378+0000 7f07b441d700 1 -- start start 2023-06-09T04:35:44.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b441d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0b1360 0x7f07ac0b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b441d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07ac0aeaa0 0x7f07ac0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b441d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac0adde0 con 0x7f07ac0ae630 2023-06-09T04:35:44.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b441d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac0ae270 con 0x7f07ac0aeaa0 2023-06-09T04:35:44.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b441d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0af0f0 con 0x7f07ac0b1360 2023-06-09T04:35:44.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b29ba700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07ac0aeaa0 0x7f07ac0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b19b8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f07ac0ae630 0x7f07ac0ac400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36824/0 (socket says 172.21.15.92:36824) 2023-06-09T04:35:44.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b19b8700 1 -- 172.21.15.92:0/1718996930 learned_addr learned my addr 172.21.15.92:0/1718996930 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:44.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b29ba700 1 -- 172.21.15.92:0/1718996930 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0b1360 msgr2=0x7f07ac0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:44.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b29ba700 1 --2- 172.21.15.92:0/1718996930 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0b1360 0x7f07ac0b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b29ba700 1 -- 172.21.15.92:0/1718996930 >> v1:172.21.15.92:6789/0 conn(0x7f07ac0ae630 legacy=0x7f07ac0ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b29ba700 1 -- 172.21.15.92:0/1718996930 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0b1c70 con 0x7f07ac0aeaa0 2023-06-09T04:35:44.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b11b7700 1 -- 172.21.15.92:0/1718996930 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312379915 0 0) 0x7f07ac0adde0 con 0x7f07ac0ae630 2023-06-09T04:35:44.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.379+0000 7f07b29ba700 1 --2- 172.21.15.92:0/1718996930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0acb10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009d90 tx=0x7f07a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbc165ce3d2cb3b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b11b7700 1 -- 172.21.15.92:0/1718996930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400e070 con 0x7f07ac0aeaa0 2023-06-09T04:35:44.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b11b7700 1 -- 172.21.15.92:0/1718996930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a400bbd0 con 0x7f07ac0aeaa0 2023-06-09T04:35:44.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b11b7700 1 -- 172.21.15.92:0/1718996930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4012430 con 0x7f07ac0aeaa0 2023-06-09T04:35:44.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 -- 172.21.15.92:0/1718996930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0aeaa0 msgr2=0x7f07ac0acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 --2- 172.21.15.92:0/1718996930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0acb10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009d90 tx=0x7f07a400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 -- 172.21.15.92:0/1718996930 shutdown_connections 2023-06-09T04:35:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 --2- 172.21.15.92:0/1718996930 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0b1360 0x7f07ac0b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 --2- 172.21.15.92:0/1718996930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0acb10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 -- 172.21.15.92:0/1718996930 >> 172.21.15.92:0/1718996930 conn(0x7f07ac098c10 msgr2=0x7f07ac0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 -- 172.21.15.92:0/1718996930 shutdown_connections 2023-06-09T04:35:44.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.380+0000 7f07b441d700 1 -- 172.21.15.92:0/1718996930 wait complete. 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.382+0000 7f07b441d700 1 Processor -- start 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.382+0000 7f07b441d700 1 -- start start 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b441d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0ae630 0x7f07ac155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b441d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b441d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0b1360 0x7f07ac0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b441d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1557d0 con 0x7f07ac0b1360 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b441d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0b49d0 con 0x7f07ac0aeaa0 2023-06-09T04:35:44.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b441d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0b53a0 con 0x7f07ac0ae630 2023-06-09T04:35:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0b1360 0x7f07ac0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0b1360 0x7f07ac0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38502/0 (socket says 172.21.15.92:38502) 2023-06-09T04:35:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 -- 172.21.15.92:0/3985054756 learned_addr learned my addr 172.21.15.92:0/3985054756 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b19b8700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b21b9700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0ae630 0x7f07ac155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 -- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0ae630 msgr2=0x7f07ac155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0ae630 0x7f07ac155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 -- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07ac0aeaa0 msgr2=0x7f07ac0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 -- 172.21.15.92:0/3985054756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4009a70 con 0x7f07ac0b1360 2023-06-09T04:35:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b19b8700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b21b9700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0ae630 0x7f07ac155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b29ba700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0b1360 0x7f07ac0cc500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f07ac070ba0 tx=0x7f07a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07a2ffd700 1 -- 172.21.15.92:0/3985054756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4012430 con 0x7f07ac0b1360 2023-06-09T04:35:44.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07a2ffd700 1 -- 172.21.15.92:0/3985054756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4020070 con 0x7f07ac0b1360 2023-06-09T04:35:44.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac0b3dc0 con 0x7f07ac0b1360 2023-06-09T04:35:44.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.383+0000 7f07a2ffd700 1 -- 172.21.15.92:0/3985054756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4004280 con 0x7f07ac0b1360 2023-06-09T04:35:44.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.384+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac0b4380 con 0x7f07ac0b1360 2023-06-09T04:35:44.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.384+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac0740e0 con 0x7f07ac0b1360 2023-06-09T04:35:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.385+0000 7f07a2ffd700 1 -- 172.21.15.92:0/3985054756 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f07a40043e0 con 0x7f07ac0b1360 2023-06-09T04:35:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.385+0000 7f07a2ffd700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f079006ace0 0x7f079006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.385+0000 7f07a2ffd700 1 -- 172.21.15.92:0/3985054756 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07a4022070 con 0x7f07ac0b1360 2023-06-09T04:35:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.385+0000 7f07b21b9700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f079006ace0 0x7f079006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.386+0000 7f07b21b9700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f079006ace0 0x7f079006d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07ac1e3fa0 tx=0x7f079c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.390+0000 7f07a2ffd700 1 -- 172.21.15.92:0/3985054756 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f07a4056860 con 0x7f07ac0b1360 2023-06-09T04:35:44.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.576+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07ac0c3160 con 0x7f07ac0b1360 2023-06-09T04:35:44.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.578+0000 7f07a2ffd700 1 -- 172.21.15.92:0/3985054756 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f07a4059e80 con 0x7f07ac0b1360 2023-06-09T04:35:44.579 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:44.579 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:44.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.579+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f079006ace0 msgr2=0x7f079006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.579+0000 7f07b441d700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f079006ace0 0x7f079006d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f07ac1e3fa0 tx=0x7f079c008040 comp rx=0 tx=0).stop 2023-06-09T04:35:44.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.579+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0b1360 msgr2=0x7f07ac0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.579+0000 7f07b441d700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0b1360 0x7f07ac0cc500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f07ac070ba0 tx=0x7f07a4004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 shutdown_connections 2023-06-09T04:35:44.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f079006ace0 0x7f079006d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07ac0b1360 0x7f07ac0cc500 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f07ac0ae630 0x7f07ac155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 --2- 172.21.15.92:0/3985054756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07ac0aeaa0 0x7f07ac0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 >> 172.21.15.92:0/3985054756 conn(0x7f07ac098c10 msgr2=0x7f07ac0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:44.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 shutdown_connections 2023-06-09T04:35:44.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.580+0000 7f07b441d700 1 -- 172.21.15.92:0/3985054756 wait complete. 2023-06-09T04:35:44.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:44.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:44.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.715+0000 7f669e53f700 1 Processor -- start 2023-06-09T04:35:44.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.715+0000 7f669e53f700 1 -- start start 2023-06-09T04:35:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f669e53f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6698069db0 0x7f669815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f669e53f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66980cf440 0x7f669806a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f669e53f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669815d040 con 0x7f66980cce60 2023-06-09T04:35:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f669e53f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6698160ab0 con 0x7f66980cf440 2023-06-09T04:35:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f669e53f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669815ca10 con 0x7f6698069db0 2023-06-09T04:35:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66980cf440 0x7f669806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66980cf440 0x7f669806a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38508/0 (socket says 172.21.15.92:38508) 2023-06-09T04:35:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 -- 172.21.15.92:0/1551373282 learned_addr learned my addr 172.21.15.92:0/1551373282 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f6697fff700 1 --2- 172.21.15.92:0/1551373282 >> v2:172.21.15.123:3300/0 conn(0x7f6698069db0 0x7f669815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 -- 172.21.15.92:0/1551373282 >> v2:172.21.15.123:3300/0 conn(0x7f6698069db0 msgr2=0x7f669815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:44.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 --2- 172.21.15.92:0/1551373282 >> v2:172.21.15.123:3300/0 conn(0x7f6698069db0 0x7f669815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 -- 172.21.15.92:0/1551373282 >> v1:172.21.15.123:6789/0 conn(0x7f66980cce60 legacy=0x7f669806ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:44.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 -- 172.21.15.92:0/1551373282 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669806a6c0 con 0x7f66980cf440 2023-06-09T04:35:44.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f6697fff700 1 --2- 172.21.15.92:0/1551373282 >> v2:172.21.15.123:3300/0 conn(0x7f6698069db0 0x7f669815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:44.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f66977fe700 1 --2- 172.21.15.92:0/1551373282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f669806a180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f66980b2430 tx=0x7f6688009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61877bd1058769b3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f6696ffd700 1 -- 172.21.15.92:0/1551373282 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688004230 con 0x7f66980cf440 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f6696ffd700 1 -- 172.21.15.92:0/1551373282 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688004390 con 0x7f66980cf440 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.716+0000 7f6696ffd700 1 -- 172.21.15.92:0/1551373282 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880036a0 con 0x7f66980cf440 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 -- 172.21.15.92:0/1551373282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 msgr2=0x7f669806a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 --2- 172.21.15.92:0/1551373282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f669806a180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f66980b2430 tx=0x7f6688009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 -- 172.21.15.92:0/1551373282 shutdown_connections 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 --2- 172.21.15.92:0/1551373282 >> v2:172.21.15.123:3300/0 conn(0x7f6698069db0 0x7f669815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 --2- 172.21.15.92:0/1551373282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f669806a180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 -- 172.21.15.92:0/1551373282 >> 172.21.15.92:0/1551373282 conn(0x7f66980cc380 msgr2=0x7f66980cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:44.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 -- 172.21.15.92:0/1551373282 shutdown_connections 2023-06-09T04:35:44.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.717+0000 7f669e53f700 1 -- 172.21.15.92:0/1551373282 wait complete. 2023-06-09T04:35:44.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.718+0000 7f669e53f700 1 Processor -- start 2023-06-09T04:35:44.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.718+0000 7f669e53f700 1 -- start start 2023-06-09T04:35:44.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669e53f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698069db0 0x7f66980b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669e53f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66980cce60 0x7f669814cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669e53f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f66980af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669e53f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980b50f0 con 0x7f66980cf440 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669e53f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980b44e0 con 0x7f6698069db0 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669e53f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66980b47c0 con 0x7f66980cce60 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669cadc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f66980af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669cadc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f66980af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38516/0 (socket says 172.21.15.92:38516) 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.719+0000 7f669cadc700 1 -- 172.21.15.92:0/2746025133 learned_addr learned my addr 172.21.15.92:0/2746025133 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f6697fff700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698069db0 0x7f66980b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f66977fe700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66980cce60 0x7f669814cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669cadc700 1 -- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66980cce60 msgr2=0x7f669814cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669cadc700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66980cce60 0x7f669814cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669cadc700 1 -- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698069db0 msgr2=0x7f66980b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669cadc700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698069db0 0x7f66980b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669cadc700 1 -- 172.21.15.92:0/2746025133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688009550 con 0x7f66980cf440 2023-06-09T04:35:44.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669cadc700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f66980af340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f669000cc10 tx=0x7f669000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f6694ff9700 1 -- 172.21.15.92:0/2746025133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669000abb0 con 0x7f66980cf440 2023-06-09T04:35:44.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f6694ff9700 1 -- 172.21.15.92:0/2746025133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669000ad10 con 0x7f66980cf440 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66980b4aa0 con 0x7f66980cf440 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f6694ff9700 1 -- 172.21.15.92:0/2746025133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690012660 con 0x7f66980cf440 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.720+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6698159cb0 con 0x7f66980cf440 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.721+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669815ccb0 con 0x7f66980cf440 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.721+0000 7f6694ff9700 1 -- 172.21.15.92:0/2746025133 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66900127c0 con 0x7f66980cf440 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.722+0000 7f6694ff9700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f668406ace0 0x7f668406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.722+0000 7f6694ff9700 1 -- 172.21.15.92:0/2746025133 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6690089b70 con 0x7f66980cf440 2023-06-09T04:35:44.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.722+0000 7f6697fff700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f668406ace0 0x7f668406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:44.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.722+0000 7f6697fff700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f668406ace0 0x7f668406d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f668c006fd0 tx=0x7f668c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:44.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.726+0000 7f6694ff9700 1 -- 172.21.15.92:0/2746025133 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f669815ccb0 con 0x7f66980cf440 2023-06-09T04:35:44.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.913+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f669815ccb0 con 0x7f66980cf440 2023-06-09T04:35:44.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.914+0000 7f6694ff9700 1 -- 172.21.15.92:0/2746025133 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f669815ccb0 con 0x7f66980cf440 2023-06-09T04:35:44.915 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:44.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f668406ace0 msgr2=0x7f668406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f668406ace0 0x7f668406d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f668c006fd0 tx=0x7f668c008040 comp rx=0 tx=0).stop 2023-06-09T04:35:44.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 msgr2=0x7f66980af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:44.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f66980af340 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f669000cc10 tx=0x7f669000a060 comp rx=0 tx=0).stop 2023-06-09T04:35:44.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 shutdown_connections 2023-06-09T04:35:44.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f668406ace0 0x7f668406d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66980cf440 0x7f66980af340 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6698069db0 0x7f66980b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 --2- 172.21.15.92:0/2746025133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66980cce60 0x7f669814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:44.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 >> 172.21.15.92:0/2746025133 conn(0x7f66980cc380 msgr2=0x7f66980cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:44.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 shutdown_connections 2023-06-09T04:35:44.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:44.916+0000 7f669e53f700 1 -- 172.21.15.92:0/2746025133 wait complete. 2023-06-09T04:35:44.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:45.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.050+0000 7f8d840a0700 1 Processor -- start 2023-06-09T04:35:45.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.050+0000 7f8d840a0700 1 -- start start 2023-06-09T04:35:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d840a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d840a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d7c15d470 0x7f8d7c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d840a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7c0ce190 con 0x7f8d7c0aed50 2023-06-09T04:35:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d840a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7c072350 con 0x7f8d7c15d470 2023-06-09T04:35:45.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d840a0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c071ab0 con 0x7f8d7c0ce3d0 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d8263d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d7c15d470 0x7f8d7c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d8163b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d7c0aed50 0x7f8d7c0ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36854/0 (socket says 172.21.15.92:36854) 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d8163b700 1 -- 172.21.15.92:0/3768938781 learned_addr learned my addr 172.21.15.92:0/3768938781 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d81e3c700 1 --2- 172.21.15.92:0/3768938781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d8263d700 1 -- 172.21.15.92:0/3768938781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 msgr2=0x7f8d7c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d8263d700 1 --2- 172.21.15.92:0/3768938781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d8263d700 1 -- 172.21.15.92:0/3768938781 >> v1:172.21.15.92:6789/0 conn(0x7f8d7c0aed50 legacy=0x7f8d7c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d80e3a700 1 -- 172.21.15.92:0/3768938781 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883892955 0 0) 0x7f8d7c0ce190 con 0x7f8d7c0aed50 2023-06-09T04:35:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.051+0000 7f8d8263d700 1 -- 172.21.15.92:0/3768938781 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c071d90 con 0x7f8d7c15d470 2023-06-09T04:35:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d8263d700 1 --2- 172.21.15.92:0/3768938781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c15d470 0x7f8d7c15d8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009d90 tx=0x7f8d7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b65933725b2b4c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d80e3a700 1 -- 172.21.15.92:0/3768938781 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400e070 con 0x7f8d7c15d470 2023-06-09T04:35:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d80e3a700 1 -- 172.21.15.92:0/3768938781 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7400bbd0 con 0x7f8d7c15d470 2023-06-09T04:35:45.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d80e3a700 1 -- 172.21.15.92:0/3768938781 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74012430 con 0x7f8d7c15d470 2023-06-09T04:35:45.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 -- 172.21.15.92:0/3768938781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c15d470 msgr2=0x7f8d7c15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 --2- 172.21.15.92:0/3768938781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c15d470 0x7f8d7c15d8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009d90 tx=0x7f8d7400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:45.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 -- 172.21.15.92:0/3768938781 shutdown_connections 2023-06-09T04:35:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 --2- 172.21.15.92:0/3768938781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 --2- 172.21.15.92:0/3768938781 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c15d470 0x7f8d7c15d8f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 -- 172.21.15.92:0/3768938781 >> 172.21.15.92:0/3768938781 conn(0x7f8d7c0cc380 msgr2=0x7f8d7c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 -- 172.21.15.92:0/3768938781 shutdown_connections 2023-06-09T04:35:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.052+0000 7f8d840a0700 1 -- 172.21.15.92:0/3768938781 wait complete. 2023-06-09T04:35:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.054+0000 7f8d840a0700 1 Processor -- start 2023-06-09T04:35:45.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.054+0000 7f8d840a0700 1 -- start start 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d840a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c0aed50 0x7f8d7c15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d840a0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d840a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c15d470 0x7f8d7c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d840a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0c9730 con 0x7f8d7c0aed50 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d840a0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0c8ab0 con 0x7f8d7c15d470 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d840a0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0c8d90 con 0x7f8d7c0ce3d0 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c0aed50 0x7f8d7c15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c0aed50 0x7f8d7c15bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38526/0 (socket says 172.21.15.92:38526) 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 -- 172.21.15.92:0/2947177076 learned_addr learned my addr 172.21.15.92:0/2947177076 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d8263d700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c15d470 0x7f8d7c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d8163b700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 -- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 msgr2=0x7f8d7c0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 -- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c15d470 msgr2=0x7f8d7c0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c15d470 0x7f8d7c0c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 -- 172.21.15.92:0/2947177076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74009a70 con 0x7f8d7c0aed50 2023-06-09T04:35:45.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d81e3c700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c0aed50 0x7f8d7c15bf00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c7f0 tx=0x7f8d6c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.055+0000 7f8d727fc700 1 -- 172.21.15.92:0/2947177076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c011800 con 0x7f8d7c0aed50 2023-06-09T04:35:45.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.056+0000 7f8d727fc700 1 -- 172.21.15.92:0/2947177076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c011960 con 0x7f8d7c0aed50 2023-06-09T04:35:45.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.056+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7c0c9070 con 0x7f8d7c0aed50 2023-06-09T04:35:45.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.056+0000 7f8d727fc700 1 -- 172.21.15.92:0/2947177076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c011cf0 con 0x7f8d7c0aed50 2023-06-09T04:35:45.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.056+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7c0c8760 con 0x7f8d7c0aed50 2023-06-09T04:35:45.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.056+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7c06be50 con 0x7f8d7c0aed50 2023-06-09T04:35:45.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.057+0000 7f8d727fc700 1 -- 172.21.15.92:0/2947177076 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d6c018360 con 0x7f8d7c0aed50 2023-06-09T04:35:45.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.057+0000 7f8d727fc700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d6006ace0 0x7f8d6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.057+0000 7f8d727fc700 1 -- 172.21.15.92:0/2947177076 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d6c089430 con 0x7f8d7c0aed50 2023-06-09T04:35:45.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.057+0000 7f8d8163b700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d6006ace0 0x7f8d6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.058+0000 7f8d8163b700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d6006ace0 0x7f8d6006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000f60 tx=0x7f8d78007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.062+0000 7f8d727fc700 1 -- 172.21.15.92:0/2947177076 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d6c057f80 con 0x7f8d7c0aed50 2023-06-09T04:35:45.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.250+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8d7c04e8e0 con 0x7f8d7c0aed50 2023-06-09T04:35:45.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.250+0000 7f8d727fc700 1 -- 172.21.15.92:0/2947177076 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8d6c057b10 con 0x7f8d7c0aed50 2023-06-09T04:35:45.252 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d6006ace0 msgr2=0x7f8d6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d6006ace0 0x7f8d6006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000f60 tx=0x7f8d78007c90 comp rx=0 tx=0).stop 2023-06-09T04:35:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c0aed50 msgr2=0x7f8d7c15bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c0aed50 0x7f8d7c15bf00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c7f0 tx=0x7f8d6c00cbb0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 shutdown_connections 2023-06-09T04:35:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8d6006ace0 0x7f8d6006d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c15d470 0x7f8d7c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d7c0aed50 0x7f8d7c15bf00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 --2- 172.21.15.92:0/2947177076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d7c0ce3d0 0x7f8d7c0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 >> 172.21.15.92:0/2947177076 conn(0x7f8d7c0cc380 msgr2=0x7f8d7c1570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 shutdown_connections 2023-06-09T04:35:45.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.252+0000 7f8d840a0700 1 -- 172.21.15.92:0/2947177076 wait complete. 2023-06-09T04:35:45.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:45.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.386+0000 7f40c4cd1700 1 Processor -- start 2023-06-09T04:35:45.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.387+0000 7f40c4cd1700 1 -- start start 2023-06-09T04:35:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.387+0000 7f40c4cd1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f40c00d03b0 0x7f40c00cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.387+0000 7f40c4cd1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c00cb700 con 0x7f40c00cd8c0 2023-06-09T04:35:45.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.387+0000 7f40c4cd1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c00cdd00 con 0x7f40c00b60d0 2023-06-09T04:35:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.387+0000 7f40c4cd1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00b64a0 con 0x7f40c00d03b0 2023-06-09T04:35:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f40c00cd8c0 0x7f40c00ce540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36870/0 (socket says 172.21.15.92:36870) 2023-06-09T04:35:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bed9d700 1 -- 172.21.15.92:0/3072546587 learned_addr learned my addr 172.21.15.92:0/3072546587 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360005568 0 0) 0x7f40b8001490 con 0x7f40c00cd8c0 2023-06-09T04:35:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/3072546587 >> v2:172.21.15.123:3300/0 conn(0x7f40c00d03b0 0x7f40c00cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a80035b0 con 0x7f40c00cd8c0 2023-06-09T04:35:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467570714 0 0) 0x7f40a80035b0 con 0x7f40c00cd8c0 2023-06-09T04:35:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 >> v2:172.21.15.123:3300/0 conn(0x7f40c00d03b0 msgr2=0x7f40c00cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 --2- 172.21.15.92:0/3072546587 >> v2:172.21.15.123:3300/0 conn(0x7f40c00d03b0 0x7f40c00cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 >> v1:172.21.15.123:6789/0 conn(0x7f40c00b60d0 legacy=0x7f40c015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00ccf50 con 0x7f40c00cd8c0 2023-06-09T04:35:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f40b8001b80 con 0x7f40c00cd8c0 2023-06-09T04:35:45.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 >> v1:172.21.15.92:6789/0 conn(0x7f40c00cd8c0 legacy=0x7f40c00ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a8003a00 0x7f40a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40a8005200 0x7f40a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00b64a0 con 0x7f40a8004850 2023-06-09T04:35:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a800c080 con 0x7f40a8003a00 2023-06-09T04:35:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a800c240 con 0x7f40a8005200 2023-06-09T04:35:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40be59c700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a8003a00 0x7f40a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.388+0000 7f40bed9d700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40a8005200 0x7f40a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bdd9b700 1 -- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40a8005200 msgr2=0x7f40a800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:45.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40a8005200 0x7f40a800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bdd9b700 1 -- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a8003a00 msgr2=0x7f40a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a8003a00 0x7f40a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bdd9b700 1 -- 172.21.15.92:0/3072546587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a80035b0 con 0x7f40a8004850 2023-06-09T04:35:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bed9d700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40a8005200 0x7f40a800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bed9d700 1 -- 172.21.15.92:0/3072546587 reap_dead start 2023-06-09T04:35:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40a8004cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009b60 tx=0x7f40a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c16c26e84138060 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000e6c0 con 0x7f40a8004850 2023-06-09T04:35:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b000e820 con 0x7f40a8004850 2023-06-09T04:35:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40bd59a700 1 -- 172.21.15.92:0/3072546587 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000eb90 con 0x7f40a8004850 2023-06-09T04:35:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40c4cd1700 1 -- 172.21.15.92:0/3072546587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 msgr2=0x7f40a8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.389+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40a8004cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009b60 tx=0x7f40a8005880 comp rx=0 tx=0).stop 2023-06-09T04:35:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.390+0000 7f40c4cd1700 1 -- 172.21.15.92:0/3072546587 shutdown_connections 2023-06-09T04:35:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.390+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/3072546587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40a8004cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.390+0000 7f40c4cd1700 1 -- 172.21.15.92:0/3072546587 >> 172.21.15.92:0/3072546587 conn(0x7f40c006e4d0 msgr2=0x7f40c006e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.390+0000 7f40c4cd1700 1 -- 172.21.15.92:0/3072546587 shutdown_connections 2023-06-09T04:35:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.390+0000 7f40c4cd1700 1 -- 172.21.15.92:0/3072546587 wait complete. 2023-06-09T04:35:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.391+0000 7f40c4cd1700 1 Processor -- start 2023-06-09T04:35:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.391+0000 7f40c4cd1700 1 -- start start 2023-06-09T04:35:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40c4cd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40c00c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40c4cd1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40c01f1110 0x7f40c013fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40c4cd1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40c00b1e70 0x7f40c014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40c4cd1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c6660 con 0x7f40a8004850 2023-06-09T04:35:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40c4cd1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c6870 con 0x7f40c00b1e70 2023-06-09T04:35:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40c4cd1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c6b50 con 0x7f40c01f1110 2023-06-09T04:35:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40be59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40c00c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40be59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40c00c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38546/0 (socket says 172.21.15.92:38546) 2023-06-09T04:35:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.392+0000 7f40be59c700 1 -- 172.21.15.92:0/1588492670 learned_addr learned my addr 172.21.15.92:0/1588492670 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40bed9d700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40c00b1e70 0x7f40c014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40be59c700 1 -- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40c01f1110 msgr2=0x7f40c013fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40c01f1110 0x7f40c013fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40be59c700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40c01f1110 0x7f40c013fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40be59c700 1 -- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40c00b1e70 msgr2=0x7f40c014edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40be59c700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40c00b1e70 0x7f40c014edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40be59c700 1 -- 172.21.15.92:0/1588492670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00c6e30 con 0x7f40a8004850 2023-06-09T04:35:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40bed9d700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40c00b1e70 0x7f40c014edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40be59c700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40c00c6120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40b400f820 tx=0x7f40b400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40aeffd700 1 -- 172.21.15.92:0/1588492670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40052e0 con 0x7f40a8004850 2023-06-09T04:35:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40aeffd700 1 -- 172.21.15.92:0/1588492670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b4005440 con 0x7f40a8004850 2023-06-09T04:35:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40aeffd700 1 -- 172.21.15.92:0/1588492670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40038c0 con 0x7f40a8004850 2023-06-09T04:35:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c00c7110 con 0x7f40a8004850 2023-06-09T04:35:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.393+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c01f1580 con 0x7f40a8004850 2023-06-09T04:35:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.394+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c01f0040 con 0x7f40a8004850 2023-06-09T04:35:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.394+0000 7f40aeffd700 1 -- 172.21.15.92:0/1588492670 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f40b4003a20 con 0x7f40a8004850 2023-06-09T04:35:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.395+0000 7f40aeffd700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f40a006ace0 0x7f40a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.395+0000 7f40aeffd700 1 -- 172.21.15.92:0/1588492670 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40b40892d0 con 0x7f40a8004850 2023-06-09T04:35:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.395+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f40a006ace0 0x7f40a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.395+0000 7f40bdd9b700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f40a006ace0 0x7f40a006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f40c00cd840 tx=0x7f40b0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.399+0000 7f40aeffd700 1 -- 172.21.15.92:0/1588492670 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f40b4054380 con 0x7f40a8004850 2023-06-09T04:35:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.587+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40c01efe80 con 0x7f40a8004850 2023-06-09T04:35:45.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.588+0000 7f40aeffd700 1 -- 172.21.15.92:0/1588492670 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40c01efe80 con 0x7f40a8004850 2023-06-09T04:35:45.590 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:45.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f40a006ace0 msgr2=0x7f40a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f40a006ace0 0x7f40a006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f40c00cd840 tx=0x7f40b0008030 comp rx=0 tx=0).stop 2023-06-09T04:35:45.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 msgr2=0x7f40c00c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40c00c6120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40b400f820 tx=0x7f40b400fbe0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 shutdown_connections 2023-06-09T04:35:45.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f40a006ace0 0x7f40a006d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40c00b1e70 0x7f40c014edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40a8004850 0x7f40c00c6120 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 --2- 172.21.15.92:0/1588492670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40c01f1110 0x7f40c013fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 >> 172.21.15.92:0/1588492670 conn(0x7f40c006e4d0 msgr2=0x7f40c00b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:45.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 shutdown_connections 2023-06-09T04:35:45.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.590+0000 7f40c4cd1700 1 -- 172.21.15.92:0/1588492670 wait complete. 2023-06-09T04:35:45.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:45.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.724+0000 7f0a582ae700 1 Processor -- start 2023-06-09T04:35:45.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.724+0000 7f0a582ae700 1 -- start start 2023-06-09T04:35:45.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.725+0000 7f0a582ae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a5015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.725+0000 7f0a582ae700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0a500b2b00 0x7f0a500ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.725+0000 7f0a582ae700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5015d270 con 0x7f0a500b2f90 2023-06-09T04:35:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.725+0000 7f0a582ae700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5015d550 con 0x7f0a500b2b00 2023-06-09T04:35:45.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.725+0000 7f0a582ae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5015d830 con 0x7f0a501569f0 2023-06-09T04:35:45.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a5684b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0a500b2b00 0x7f0a500ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a5604a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a5015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a55849700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0a500b2f90 0x7f0a500b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59382/0 (socket says 172.21.15.92:59382) 2023-06-09T04:35:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a55849700 1 -- 172.21.15.92:0/359945473 learned_addr learned my addr 172.21.15.92:0/359945473 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a55048700 1 -- 172.21.15.92:0/359945473 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77813188 0 0) 0x7f0a5015d270 con 0x7f0a500b2f90 2023-06-09T04:35:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a55048700 1 -- 172.21.15.92:0/359945473 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a3c0035b0 con 0x7f0a500b2f90 2023-06-09T04:35:45.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a5684b700 1 -- 172.21.15.92:0/359945473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 msgr2=0x7f0a5015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a5684b700 1 --2- 172.21.15.92:0/359945473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a5015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a5684b700 1 -- 172.21.15.92:0/359945473 >> v1:172.21.15.123:6789/0 conn(0x7f0a500b2f90 legacy=0x7f0a500b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a5684b700 1 -- 172.21.15.92:0/359945473 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5006af60 con 0x7f0a500b2b00 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a5684b700 1 --2- 172.21.15.92:0/359945473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2b00 0x7f0a500ccfe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009d90 tx=0x7f0a4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa277f8365be8d7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a55048700 1 -- 172.21.15.92:0/359945473 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800e070 con 0x7f0a500b2b00 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a55048700 1 -- 172.21.15.92:0/359945473 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4800bb20 con 0x7f0a500b2b00 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.726+0000 7f0a55048700 1 -- 172.21.15.92:0/359945473 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800be90 con 0x7f0a500b2b00 2023-06-09T04:35:45.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 -- 172.21.15.92:0/359945473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2b00 msgr2=0x7f0a500ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 --2- 172.21.15.92:0/359945473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2b00 0x7f0a500ccfe0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009d90 tx=0x7f0a4800b330 comp rx=0 tx=0).stop 2023-06-09T04:35:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 -- 172.21.15.92:0/359945473 shutdown_connections 2023-06-09T04:35:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 --2- 172.21.15.92:0/359945473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2b00 0x7f0a500ccfe0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 --2- 172.21.15.92:0/359945473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a5015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 -- 172.21.15.92:0/359945473 >> 172.21.15.92:0/359945473 conn(0x7f0a5006e4d0 msgr2=0x7f0a5006e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 -- 172.21.15.92:0/359945473 shutdown_connections 2023-06-09T04:35:45.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.727+0000 7f0a582ae700 1 -- 172.21.15.92:0/359945473 wait complete. 2023-06-09T04:35:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.728+0000 7f0a582ae700 1 Processor -- start 2023-06-09T04:35:45.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.728+0000 7f0a582ae700 1 -- start start 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a582ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a500b2b00 0x7f0a500c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a582ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2f90 0x7f0a50147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a582ae700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a50158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a582ae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a501f6560 con 0x7f0a500b2b00 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a582ae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a500b4620 con 0x7f0a500b2f90 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a582ae700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a501f57a0 con 0x7f0a501569f0 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a500b2b00 0x7f0a500c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a500b2b00 0x7f0a500c44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38556/0 (socket says 172.21.15.92:38556) 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 -- 172.21.15.92:0/1213707731 learned_addr learned my addr 172.21.15.92:0/1213707731 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a55849700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2f90 0x7f0a50147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5684b700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a50158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 -- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 msgr2=0x7f0a50158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a50158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 -- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2f90 msgr2=0x7f0a50147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2f90 0x7f0a50147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 -- 172.21.15.92:0/1213707731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48009a70 con 0x7f0a500b2b00 2023-06-09T04:35:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a55849700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2f90 0x7f0a50147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.729+0000 7f0a5604a700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a500b2b00 0x7f0a500c44a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000e870 tx=0x7f0a4000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.730+0000 7f0a46ffd700 1 -- 172.21.15.92:0/1213707731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40009590 con 0x7f0a500b2b00 2023-06-09T04:35:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.730+0000 7f0a46ffd700 1 -- 172.21.15.92:0/1213707731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000ee70 con 0x7f0a500b2b00 2023-06-09T04:35:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.730+0000 7f0a46ffd700 1 -- 172.21.15.92:0/1213707731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000ca50 con 0x7f0a500b2b00 2023-06-09T04:35:45.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.730+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a501f5a80 con 0x7f0a500b2b00 2023-06-09T04:35:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.730+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a501f6040 con 0x7f0a500b2b00 2023-06-09T04:35:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.731+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a501f63b0 con 0x7f0a500b2b00 2023-06-09T04:35:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.731+0000 7f0a46ffd700 1 -- 172.21.15.92:0/1213707731 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0a4000cbb0 con 0x7f0a500b2b00 2023-06-09T04:35:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.731+0000 7f0a46ffd700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0a3406ace0 0x7f0a3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.732+0000 7f0a46ffd700 1 -- 172.21.15.92:0/1213707731 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a400893c0 con 0x7f0a500b2b00 2023-06-09T04:35:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.732+0000 7f0a55849700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0a3406ace0 0x7f0a3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:45.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.732+0000 7f0a55849700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0a3406ace0 0x7f0a3406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a50154cc0 tx=0x7f0a4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:45.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.736+0000 7f0a46ffd700 1 -- 172.21.15.92:0/1213707731 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0a40057f90 con 0x7f0a500b2b00 2023-06-09T04:35:45.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.923+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a5004e0e0 con 0x7f0a500b2b00 2023-06-09T04:35:45.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.924+0000 7f0a46ffd700 1 -- 172.21.15.92:0/1213707731 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a4000ce60 con 0x7f0a500b2b00 2023-06-09T04:35:45.925 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:45.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0a3406ace0 msgr2=0x7f0a3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0a3406ace0 0x7f0a3406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a50154cc0 tx=0x7f0a4c009040 comp rx=0 tx=0).stop 2023-06-09T04:35:45.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a500b2b00 msgr2=0x7f0a500c44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:45.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a500b2b00 0x7f0a500c44a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000e870 tx=0x7f0a4000ec30 comp rx=0 tx=0).stop 2023-06-09T04:35:45.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 shutdown_connections 2023-06-09T04:35:45.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0a3406ace0 0x7f0a3406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a501569f0 0x7f0a50158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a500b2b00 0x7f0a500c44a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 --2- 172.21.15.92:0/1213707731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a500b2f90 0x7f0a50147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 >> 172.21.15.92:0/1213707731 conn(0x7f0a5006e4d0 msgr2=0x7f0a5015b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 shutdown_connections 2023-06-09T04:35:45.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:45.926+0000 7f0a582ae700 1 -- 172.21.15.92:0/1213707731 wait complete. 2023-06-09T04:35:45.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:45.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:35:45.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:35:46.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.062+0000 7ff86bea7700 1 Processor -- start 2023-06-09T04:35:46.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.062+0000 7ff86bea7700 1 -- start start 2023-06-09T04:35:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff86bea7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 0x7ff8640ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff86bea7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff8640ac400 0x7ff86406e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff86bea7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8640ce730 con 0x7ff86415c7d0 2023-06-09T04:35:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff86bea7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff86406aaf0 con 0x7ff8640ac400 2023-06-09T04:35:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff86bea7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86415cba0 con 0x7ff8640cf010 2023-06-09T04:35:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff869c43700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff86415c7d0 0x7ff8640ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36896/0 (socket says 172.21.15.92:36896) 2023-06-09T04:35:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff869c43700 1 -- 172.21.15.92:0/1730342303 learned_addr learned my addr 172.21.15.92:0/1730342303 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff869442700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 0x7ff8640ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff86a444700 1 --2- 172.21.15.92:0/1730342303 >> v2:172.21.15.123:3300/0 conn(0x7ff8640ac400 0x7ff86406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35250023 0 0) 0x7ff854001290 con 0x7ff86415c7d0 2023-06-09T04:35:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.063+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8500035b0 con 0x7ff86415c7d0 2023-06-09T04:35:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2262147602 0 0) 0x7ff8500035b0 con 0x7ff86415c7d0 2023-06-09T04:35:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 msgr2=0x7ff8640ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 0x7ff8640ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 >> v2:172.21.15.123:3300/0 conn(0x7ff8640ac400 msgr2=0x7ff86406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 --2- 172.21.15.92:0/1730342303 >> v2:172.21.15.123:3300/0 conn(0x7ff8640ac400 0x7ff86406e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff864069ee0 con 0x7ff86415c7d0 2023-06-09T04:35:46.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7ff854001980 con 0x7ff86415c7d0 2023-06-09T04:35:46.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 >> v1:172.21.15.92:6789/0 conn(0x7ff86415c7d0 legacy=0x7ff8640ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 0x7ff850003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850003f80 0x7ff850004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff850004d90 0x7ff850005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86406aaf0 con 0x7ff850004d90 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86415cba0 con 0x7ff850003f80 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85000ba40 con 0x7ff8640cf010 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff86a444700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff850004d90 0x7ff850005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff869c43700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 0x7ff850003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff869442700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850003f80 0x7ff850004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff86a444700 1 -- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 msgr2=0x7ff850003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff86a444700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640cf010 0x7ff850003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff86a444700 1 -- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850003f80 msgr2=0x7ff850004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff86a444700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850003f80 0x7ff850004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff86a444700 1 -- 172.21.15.92:0/1730342303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500035b0 con 0x7ff850004d90 2023-06-09T04:35:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff869442700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850003f80 0x7ff850004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff86a444700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff850004d90 0x7ff850005200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009f90 tx=0x7ff850005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3fb6be90f3bbc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00ebd0 con 0x7ff850004d90 2023-06-09T04:35:46.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c00f040 con 0x7ff850004d90 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.064+0000 7ff868c41700 1 -- 172.21.15.92:0/1730342303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c014480 con 0x7ff850004d90 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.065+0000 7ff86bea7700 1 -- 172.21.15.92:0/1730342303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff850004d90 msgr2=0x7ff850005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.065+0000 7ff86bea7700 1 --2- 172.21.15.92:0/1730342303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff850004d90 0x7ff850005200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009f90 tx=0x7ff850005880 comp rx=0 tx=0).stop 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.065+0000 7ff86a444700 1 -- 172.21.15.92:0/1730342303 reap_dead start 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.065+0000 7ff86bea7700 1 -- 172.21.15.92:0/1730342303 shutdown_connections 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.065+0000 7ff86bea7700 1 -- 172.21.15.92:0/1730342303 >> 172.21.15.92:0/1730342303 conn(0x7ff864098c10 msgr2=0x7ff8640cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.065+0000 7ff86bea7700 1 -- 172.21.15.92:0/1730342303 shutdown_connections 2023-06-09T04:35:46.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.065+0000 7ff86bea7700 1 -- 172.21.15.92:0/1730342303 wait complete. 2023-06-09T04:35:46.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.066+0000 7ff86bea7700 1 Processor -- start 2023-06-09T04:35:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.066+0000 7ff86bea7700 1 -- start start 2023-06-09T04:35:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86bea7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850004d90 0x7ff86414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86bea7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640c3380 0x7ff86409a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86bea7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8640c4920 0x7ff8640c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86bea7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640c4500 con 0x7ff8640c4920 2023-06-09T04:35:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86bea7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640c3880 con 0x7ff850004d90 2023-06-09T04:35:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86bea7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8640c3b60 con 0x7ff8640c3380 2023-06-09T04:35:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8640c4920 0x7ff8640c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8640c4920 0x7ff8640c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38580/0 (socket says 172.21.15.92:38580) 2023-06-09T04:35:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 -- 172.21.15.92:0/2312353614 learned_addr learned my addr 172.21.15.92:0/2312353614 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff869442700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640c3380 0x7ff86409a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 -- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640c3380 msgr2=0x7ff86409a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640c3380 0x7ff86409a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 -- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850004d90 msgr2=0x7ff86414bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff869c43700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850004d90 0x7ff86414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850004d90 0x7ff86414bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 -- 172.21.15.92:0/2312353614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8640c3e40 con 0x7ff8640c4920 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.067+0000 7ff86a444700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8640c4920 0x7ff8640c59c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff85c002230 tx=0x7ff85c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.068+0000 7ff85a7fc700 1 -- 172.21.15.92:0/2312353614 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c007000 con 0x7ff8640c4920 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.068+0000 7ff85a7fc700 1 -- 172.21.15.92:0/2312353614 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85c003e50 con 0x7ff8640c4920 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.068+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8640c4120 con 0x7ff8640c4920 2023-06-09T04:35:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.068+0000 7ff85a7fc700 1 -- 172.21.15.92:0/2312353614 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c015b60 con 0x7ff8640c4920 2023-06-09T04:35:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.068+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86409a700 con 0x7ff8640c4920 2023-06-09T04:35:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.069+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8640c4640 con 0x7ff8640c4920 2023-06-09T04:35:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.069+0000 7ff85a7fc700 1 -- 172.21.15.92:0/2312353614 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff85c015cc0 con 0x7ff8640c4920 2023-06-09T04:35:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.069+0000 7ff85a7fc700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff84806ace0 0x7ff84806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.070+0000 7ff85a7fc700 1 -- 172.21.15.92:0/2312353614 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff85c08a800 con 0x7ff8640c4920 2023-06-09T04:35:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.070+0000 7ff869c43700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff84806ace0 0x7ff84806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.070+0000 7ff869c43700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff84806ace0 0x7ff84806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff854001130 tx=0x7ff854009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.074+0000 7ff85a7fc700 1 -- 172.21.15.92:0/2312353614 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff8640c4640 con 0x7ff8640c4920 2023-06-09T04:35:46.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.263+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8640c4640 con 0x7ff8640c4920 2023-06-09T04:35:46.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.264+0000 7ff85a7fc700 1 -- 172.21.15.92:0/2312353614 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7ff8640c4640 con 0x7ff8640c4920 2023-06-09T04:35:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:46.266 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:46.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff84806ace0 msgr2=0x7ff84806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff84806ace0 0x7ff84806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff854001130 tx=0x7ff854009040 comp rx=0 tx=0).stop 2023-06-09T04:35:46.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8640c4920 msgr2=0x7ff8640c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8640c4920 0x7ff8640c59c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff85c002230 tx=0x7ff85c009750 comp rx=0 tx=0).stop 2023-06-09T04:35:46.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 shutdown_connections 2023-06-09T04:35:46.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff84806ace0 0x7ff84806d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8640c4920 0x7ff8640c59c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850004d90 0x7ff86414bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 --2- 172.21.15.92:0/2312353614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8640c3380 0x7ff86409a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 >> 172.21.15.92:0/2312353614 conn(0x7ff864098c10 msgr2=0x7ff8640b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 shutdown_connections 2023-06-09T04:35:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.266+0000 7ff86bea7700 1 -- 172.21.15.92:0/2312353614 wait complete. 2023-06-09T04:35:46.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:46.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:46.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.440+0000 7f13c1898700 1 Processor -- start 2023-06-09T04:35:46.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.440+0000 7f13c1898700 1 -- start start 2023-06-09T04:35:46.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.441+0000 7f13c1898700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 0x7f13bc0d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.441+0000 7f13c1898700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13bc0d2b00 con 0x7f13bc15ced0 2023-06-09T04:35:46.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.441+0000 7f13c1898700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13bc0cd3c0 con 0x7f13bc0d3800 2023-06-09T04:35:46.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.441+0000 7f13c1898700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc0cd6a0 con 0x7f13bc0cf100 2023-06-09T04:35:46.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.441+0000 7f13baffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f13bc15ced0 0x7f13bc15d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36916/0 (socket says 172.21.15.92:36916) 2023-06-09T04:35:46.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.441+0000 7f13baffd700 1 -- 172.21.15.92:0/561902477 learned_addr learned my addr 172.21.15.92:0/561902477 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203049961 0 0) 0x7f13bc0d2b00 con 0x7f13bc15ced0 2023-06-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13a40035b0 con 0x7f13bc15ced0 2023-06-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13ba7fc700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 0x7f13bc0d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2541470493 0 0) 0x7f13a40035b0 con 0x7f13bc15ced0 2023-06-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 msgr2=0x7f13bc0d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 0x7f13bc0d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 >> v1:172.21.15.123:6789/0 conn(0x7f13bc0d3800 legacy=0x7f13bc0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc0d2da0 con 0x7f13bc15ced0 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f13b0001670 con 0x7f13bc15ced0 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 >> v1:172.21.15.92:6789/0 conn(0x7f13bc15ced0 legacy=0x7f13bc15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4003a30 0x7f13a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 0x7f13a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13a4004d90 0x7f13a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc0cd6a0 con 0x7f13a4004d90 2023-06-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a400bb20 con 0x7f13a4003a30 2023-06-09T04:35:46.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a400bce0 con 0x7f13bc0cf100 2023-06-09T04:35:46.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13bb7fe700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13a4004d90 0x7f13a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13baffd700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4003a30 0x7f13a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13ba7fc700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 0x7f13a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13bb7fe700 1 -- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 msgr2=0x7f13a4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13bb7fe700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc0cf100 0x7f13a4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13bb7fe700 1 -- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4003a30 msgr2=0x7f13a4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13bb7fe700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4003a30 0x7f13a4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.442+0000 7f13bb7fe700 1 -- 172.21.15.92:0/561902477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40035b0 con 0x7f13a4004d90 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13bb7fe700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13a4004d90 0x7f13a4005200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c160 tx=0x7f13a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1a3054124da2c4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4001db0 con 0x7f13a4004d90 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b400cca0 con 0x7f13a4004d90 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13b9ffb700 1 -- 172.21.15.92:0/561902477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4003c40 con 0x7f13a4004d90 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13c1898700 1 -- 172.21.15.92:0/561902477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13a4004d90 msgr2=0x7f13a4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13c1898700 1 --2- 172.21.15.92:0/561902477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13a4004d90 0x7f13a4005200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c160 tx=0x7f13a4005880 comp rx=0 tx=0).stop 2023-06-09T04:35:46.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13bb7fe700 1 -- 172.21.15.92:0/561902477 reap_dead start 2023-06-09T04:35:46.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13c1898700 1 -- 172.21.15.92:0/561902477 shutdown_connections 2023-06-09T04:35:46.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13c1898700 1 -- 172.21.15.92:0/561902477 >> 172.21.15.92:0/561902477 conn(0x7f13bc06b2e0 msgr2=0x7f13bc06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:46.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13c1898700 1 -- 172.21.15.92:0/561902477 shutdown_connections 2023-06-09T04:35:46.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.443+0000 7f13c1898700 1 -- 172.21.15.92:0/561902477 wait complete. 2023-06-09T04:35:46.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.444+0000 7f13c1898700 1 Processor -- start 2023-06-09T04:35:46.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13c1898700 1 -- start start 2023-06-09T04:35:46.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13c1898700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4004d90 0x7f13bc0b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13c1898700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc153a50 0x7f13bc1f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13c1898700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc14cc90 0x7f13bc14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13c1898700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc154dd0 con 0x7f13bc153a50 2023-06-09T04:35:46.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13c1898700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc14d620 con 0x7f13a4004d90 2023-06-09T04:35:46.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13c1898700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc1541c0 con 0x7f13bc14cc90 2023-06-09T04:35:46.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.445+0000 7f13ba7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc153a50 0x7f13bc1f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc153a50 0x7f13bc1f5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38598/0 (socket says 172.21.15.92:38598) 2023-06-09T04:35:46.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 -- 172.21.15.92:0/757853562 learned_addr learned my addr 172.21.15.92:0/757853562 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:46.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13bb7fe700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc14cc90 0x7f13bc14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13baffd700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4004d90 0x7f13bc0b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 -- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc14cc90 msgr2=0x7f13bc14d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc14cc90 0x7f13bc14d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 -- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4004d90 msgr2=0x7f13bc0b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4004d90 0x7f13bc0b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 -- 172.21.15.92:0/757853562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc1544a0 con 0x7f13bc153a50 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13ba7fc700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc153a50 0x7f13bc1f5720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0116c0 tx=0x7f13ac0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13a3fff700 1 -- 172.21.15.92:0/757853562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00c070 con 0x7f13bc153a50 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13a3fff700 1 -- 172.21.15.92:0/757853562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac015870 con 0x7f13bc153a50 2023-06-09T04:35:46.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13bc154780 con 0x7f13bc153a50 2023-06-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13a3fff700 1 -- 172.21.15.92:0/757853562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac015c20 con 0x7f13bc153a50 2023-06-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.446+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13bc153ec0 con 0x7f13bc153a50 2023-06-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.447+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13bc04e8e0 con 0x7f13bc153a50 2023-06-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.447+0000 7f13a3fff700 1 -- 172.21.15.92:0/757853562 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13ac0143e0 con 0x7f13bc153a50 2023-06-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.448+0000 7f13a3fff700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f13a806ace0 0x7f13a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.448+0000 7f13baffd700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f13a806ace0 0x7f13a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.448+0000 7f13a3fff700 1 -- 172.21.15.92:0/757853562 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13ac0895e0 con 0x7f13bc153a50 2023-06-09T04:35:46.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.448+0000 7f13baffd700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f13a806ace0 0x7f13a806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f13b0001c90 tx=0x7f13b0001c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.452+0000 7f13a3fff700 1 -- 172.21.15.92:0/757853562 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f13ac054690 con 0x7f13bc153a50 2023-06-09T04:35:46.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.637+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13bc0b5640 con 0x7f13bc153a50 2023-06-09T04:35:46.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.638+0000 7f13a3fff700 1 -- 172.21.15.92:0/757853562 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13ac057cb0 con 0x7f13bc153a50 2023-06-09T04:35:46.639 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:46.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f13a806ace0 msgr2=0x7f13a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f13a806ace0 0x7f13a806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f13b0001c90 tx=0x7f13b0001c00 comp rx=0 tx=0).stop 2023-06-09T04:35:46.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc153a50 msgr2=0x7f13bc1f5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc153a50 0x7f13bc1f5720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0116c0 tx=0x7f13ac0119d0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 shutdown_connections 2023-06-09T04:35:46.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f13a806ace0 0x7f13a806d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13bc14cc90 0x7f13bc14d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13a4004d90 0x7f13bc0b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 --2- 172.21.15.92:0/757853562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13bc153a50 0x7f13bc1f5720 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 >> 172.21.15.92:0/757853562 conn(0x7f13bc06b2e0 msgr2=0x7f13bc0b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 shutdown_connections 2023-06-09T04:35:46.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.640+0000 7f13c1898700 1 -- 172.21.15.92:0/757853562 wait complete. 2023-06-09T04:35:46.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:46.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.773+0000 7fa4a903e700 1 Processor -- start 2023-06-09T04:35:46.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.773+0000 7fa4a903e700 1 -- start start 2023-06-09T04:35:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.774+0000 7fa4a903e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40cb950 0x7fa4a40cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.774+0000 7fa4a903e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4a41607b0 0x7fa4a40c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.774+0000 7fa4a903e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4a40c5be0 0x7fa4a4070f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.774+0000 7fa4a903e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a4070a10 con 0x7fa4a40c5be0 2023-06-09T04:35:46.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.774+0000 7fa4a903e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a40cc260 con 0x7fa4a41607b0 2023-06-09T04:35:46.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.774+0000 7fa4a903e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a4160c90 con 0x7fa4a40cb950 2023-06-09T04:35:46.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.774+0000 7fa4a359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4a40c5be0 0x7fa4a4070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa4a40c5be0 0x7fa4a4070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38608/0 (socket says 172.21.15.92:38608) 2023-06-09T04:35:46.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 -- 172.21.15.92:0/4191627602 learned_addr learned my addr 172.21.15.92:0/4191627602 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:46.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a2d9d700 1 --2- 172.21.15.92:0/4191627602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40cb950 0x7fa4a40cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 -- 172.21.15.92:0/4191627602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40cb950 msgr2=0x7fa4a40cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 --2- 172.21.15.92:0/4191627602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40cb950 0x7fa4a40cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 -- 172.21.15.92:0/4191627602 >> v2:172.21.15.123:3300/0 conn(0x7fa4a41607b0 msgr2=0x7fa4a40c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a259c700 1 --2- 172.21.15.92:0/4191627602 >> v2:172.21.15.123:3300/0 conn(0x7fa4a41607b0 0x7fa4a40c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 --2- 172.21.15.92:0/4191627602 >> v2:172.21.15.123:3300/0 conn(0x7fa4a41607b0 0x7fa4a40c95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 -- 172.21.15.92:0/4191627602 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4065af0 con 0x7fa4a40c5be0 2023-06-09T04:35:46.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a359e700 1 --2- 172.21.15.92:0/4191627602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a40c5be0 0x7fa4a4070f10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4070520 tx=0x7fa49c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94c8fe95dc974f8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a1d9b700 1 -- 172.21.15.92:0/4191627602 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00e070 con 0x7fa4a40c5be0 2023-06-09T04:35:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a1d9b700 1 -- 172.21.15.92:0/4191627602 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c00b9c0 con 0x7fa4a40c5be0 2023-06-09T04:35:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.775+0000 7fa4a1d9b700 1 -- 172.21.15.92:0/4191627602 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00bd70 con 0x7fa4a40c5be0 2023-06-09T04:35:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 -- 172.21.15.92:0/4191627602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a40c5be0 msgr2=0x7fa4a4070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 --2- 172.21.15.92:0/4191627602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a40c5be0 0x7fa4a4070f10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4070520 tx=0x7fa49c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 -- 172.21.15.92:0/4191627602 shutdown_connections 2023-06-09T04:35:46.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 --2- 172.21.15.92:0/4191627602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a40c5be0 0x7fa4a4070f10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 --2- 172.21.15.92:0/4191627602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40cb950 0x7fa4a40cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 --2- 172.21.15.92:0/4191627602 >> v2:172.21.15.123:3300/0 conn(0x7fa4a41607b0 0x7fa4a40c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 -- 172.21.15.92:0/4191627602 >> 172.21.15.92:0/4191627602 conn(0x7fa4a406d820 msgr2=0x7fa4a40cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 -- 172.21.15.92:0/4191627602 shutdown_connections 2023-06-09T04:35:46.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.776+0000 7fa4a903e700 1 -- 172.21.15.92:0/4191627602 wait complete. 2023-06-09T04:35:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.777+0000 7fa4a903e700 1 Processor -- start 2023-06-09T04:35:46.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.777+0000 7fa4a903e700 1 -- start start 2023-06-09T04:35:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a903e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40c5be0 0x7fa4a40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a903e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a40cb950 0x7fa4a41f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a903e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a41607b0 0x7fa4a40add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a903e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40b3aa0 con 0x7fa4a41607b0 2023-06-09T04:35:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a903e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40b3d80 con 0x7fa4a40cb950 2023-06-09T04:35:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a903e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40b4060 con 0x7fa4a40c5be0 2023-06-09T04:35:46.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a41607b0 0x7fa4a40add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a41607b0 0x7fa4a40add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38614/0 (socket says 172.21.15.92:38614) 2023-06-09T04:35:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 -- 172.21.15.92:0/2252205469 learned_addr learned my addr 172.21.15.92:0/2252205469 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a2d9d700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40c5be0 0x7fa4a40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a259c700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a40cb950 0x7fa4a41f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 -- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40c5be0 msgr2=0x7fa4a40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40c5be0 0x7fa4a40b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 -- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a40cb950 msgr2=0x7fa4a41f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a40cb950 0x7fa4a41f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.778+0000 7fa4a359e700 1 -- 172.21.15.92:0/2252205469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c009b30 con 0x7fa4a41607b0 2023-06-09T04:35:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.779+0000 7fa4a359e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a41607b0 0x7fa4a40add40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b370 tx=0x7fa49c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.779+0000 7fa48b7fe700 1 -- 172.21.15.92:0/2252205469 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c004310 con 0x7fa4a41607b0 2023-06-09T04:35:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.779+0000 7fa48b7fe700 1 -- 172.21.15.92:0/2252205469 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49c004470 con 0x7fa4a41607b0 2023-06-09T04:35:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.779+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a40b4340 con 0x7fa4a41607b0 2023-06-09T04:35:46.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.779+0000 7fa48b7fe700 1 -- 172.21.15.92:0/2252205469 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49c004820 con 0x7fa4a41607b0 2023-06-09T04:35:46.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.779+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a40b36d0 con 0x7fa4a41607b0 2023-06-09T04:35:46.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.780+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a40b3860 con 0x7fa4a41607b0 2023-06-09T04:35:46.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.780+0000 7fa48b7fe700 1 -- 172.21.15.92:0/2252205469 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa49c01b430 con 0x7fa4a41607b0 2023-06-09T04:35:46.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.780+0000 7fa48b7fe700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa49006ace0 0x7fa49006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:46.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.781+0000 7fa48b7fe700 1 -- 172.21.15.92:0/2252205469 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa49c08a6e0 con 0x7fa4a41607b0 2023-06-09T04:35:46.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.781+0000 7fa4a2d9d700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa49006ace0 0x7fa49006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:46.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.781+0000 7fa4a2d9d700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa49006ace0 0x7fa49006d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa49800a970 tx=0x7fa498008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:46.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.785+0000 7fa48b7fe700 1 -- 172.21.15.92:0/2252205469 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa4a40b3860 con 0x7fa4a41607b0 2023-06-09T04:35:46.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.972+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa4a40b4860 con 0x7fa4a41607b0 2023-06-09T04:35:46.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.973+0000 7fa48b7fe700 1 -- 172.21.15.92:0/2252205469 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa4a40b4860 con 0x7fa4a41607b0 2023-06-09T04:35:46.974 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa49006ace0 msgr2=0x7fa49006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa49006ace0 0x7fa49006d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa49800a970 tx=0x7fa498008040 comp rx=0 tx=0).stop 2023-06-09T04:35:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a41607b0 msgr2=0x7fa4a40add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a41607b0 0x7fa4a40add40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00b370 tx=0x7fa49c004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 shutdown_connections 2023-06-09T04:35:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa49006ace0 0x7fa49006d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa4a41607b0 0x7fa4a40add40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4a40c5be0 0x7fa4a40b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 --2- 172.21.15.92:0/2252205469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4a40cb950 0x7fa4a41f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 >> 172.21.15.92:0/2252205469 conn(0x7fa4a406d820 msgr2=0x7fa4a40ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 shutdown_connections 2023-06-09T04:35:46.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:46.975+0000 7fa4a903e700 1 -- 172.21.15.92:0/2252205469 wait complete. 2023-06-09T04:35:46.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.111+0000 7fd90b469700 1 Processor -- start 2023-06-09T04:35:47.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.112+0000 7fd90b469700 1 -- start start 2023-06-09T04:35:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.112+0000 7fd90b469700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd9040ce3d0 0x7fd9040713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.112+0000 7fd90b469700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd90415d470 0x7fd90415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.112+0000 7fd90b469700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9040ce190 con 0x7fd9040aed50 2023-06-09T04:35:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.112+0000 7fd90b469700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd904072350 con 0x7fd9040ce3d0 2023-06-09T04:35:47.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.112+0000 7fd90b469700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd904071ab0 con 0x7fd90415d470 2023-06-09T04:35:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd908a04700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd9040aed50 0x7fd9040ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36952/0 (socket says 172.21.15.92:36952) 2023-06-09T04:35:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd908a04700 1 -- 172.21.15.92:0/1320409419 learned_addr learned my addr 172.21.15.92:0/1320409419 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd909205700 1 --2- 172.21.15.92:0/1320409419 >> v2:172.21.15.123:3300/0 conn(0x7fd9040ce3d0 0x7fd9040713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd909a06700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd90415d470 0x7fd90415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746629057 0 0) 0x7fd9040ce190 con 0x7fd9040aed50 2023-06-09T04:35:47.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f00035b0 con 0x7fd9040aed50 2023-06-09T04:35:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3464276369 0 0) 0x7fd8f00035b0 con 0x7fd9040aed50 2023-06-09T04:35:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd90415d470 msgr2=0x7fd90415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:47.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd90415d470 0x7fd90415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 >> v2:172.21.15.123:3300/0 conn(0x7fd9040ce3d0 msgr2=0x7fd9040713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 --2- 172.21.15.92:0/1320409419 >> v2:172.21.15.123:3300/0 conn(0x7fd9040ce3d0 0x7fd9040713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904071d90 con 0x7fd9040aed50 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fd900001670 con 0x7fd9040aed50 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 >> v1:172.21.15.92:6789/0 conn(0x7fd9040aed50 legacy=0x7fd9040ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0003a00 0x7fd8f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f0004430 0x7fd8f00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f0004f60 0x7fd8f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904072350 con 0x7fd8f0003a00 2023-06-09T04:35:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904071ab0 con 0x7fd8f0004430 2023-06-09T04:35:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f000bf50 con 0x7fd8f0004f60 2023-06-09T04:35:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd909205700 1 --2- 172.21.15.92:0/1320409419 >> v2:172.21.15.123:3300/0 conn(0x7fd9040ce3d0 0x7fd9040713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd909a06700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd90415d470 0x7fd90415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd908a04700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f0004430 0x7fd8f00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.113+0000 7fd909205700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0003a00 0x7fd8f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd909a06700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f0004f60 0x7fd8f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd908a04700 1 -- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f0004f60 msgr2=0x7fd8f000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd908a04700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f0004f60 0x7fd8f000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd908a04700 1 -- 172.21.15.92:0/1320409419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0003a00 msgr2=0x7fd8f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd908a04700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0003a00 0x7fd8f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd908a04700 1 -- 172.21.15.92:0/1320409419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00035b0 con 0x7fd8f0004430 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd909205700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0003a00 0x7fd8f0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd908a04700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f0004430 0x7fd8f00048f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c320 tx=0x7fd8f0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188ce9233408e999 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000ce60 con 0x7fd8f0004430 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd900005960 con 0x7fd8f0004430 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.114+0000 7fd8fbfff700 1 -- 172.21.15.92:0/1320409419 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900005cf0 con 0x7fd8f0004430 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.115+0000 7fd90b469700 1 -- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f0004430 msgr2=0x7fd8f00048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.115+0000 7fd90b469700 1 --2- 172.21.15.92:0/1320409419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f0004430 0x7fd8f00048f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd90000c320 tx=0x7fd8f0005880 comp rx=0 tx=0).stop 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.115+0000 7fd909a06700 1 -- 172.21.15.92:0/1320409419 reap_dead start 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.115+0000 7fd90b469700 1 -- 172.21.15.92:0/1320409419 shutdown_connections 2023-06-09T04:35:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.115+0000 7fd90b469700 1 -- 172.21.15.92:0/1320409419 >> 172.21.15.92:0/1320409419 conn(0x7fd9040cc380 msgr2=0x7fd9040cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:47.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.115+0000 7fd90b469700 1 -- 172.21.15.92:0/1320409419 shutdown_connections 2023-06-09T04:35:47.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.115+0000 7fd90b469700 1 -- 172.21.15.92:0/1320409419 wait complete. 2023-06-09T04:35:47.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.116+0000 7fd90b469700 1 Processor -- start 2023-06-09T04:35:47.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.116+0000 7fd90b469700 1 -- start start 2023-06-09T04:35:47.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd90b469700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0004430 0x7fd9040b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd90b469700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd904150b10 0x7fd9040c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd90b469700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9040c8860 0x7fd904069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd90b469700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9040b2aa0 con 0x7fd8f0004430 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd90b469700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9040b3c30 con 0x7fd9040c8860 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd90b469700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9040b3f10 con 0x7fd904150b10 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0004430 0x7fd9040b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0004430 0x7fd9040b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38630/0 (socket says 172.21.15.92:38630) 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 -- 172.21.15.92:0/1891654327 learned_addr learned my addr 172.21.15.92:0/1891654327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd908a04700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd904150b10 0x7fd9040c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909a06700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9040c8860 0x7fd904069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 -- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd904150b10 msgr2=0x7fd9040c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd904150b10 0x7fd9040c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 -- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9040c8860 msgr2=0x7fd904069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9040c8860 0x7fd904069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 -- 172.21.15.92:0/1891654327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9040b41f0 con 0x7fd8f0004430 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909a06700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9040c8860 0x7fd904069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd909205700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0004430 0x7fd9040b2f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4011ca0 tx=0x7fd8f400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd8f9ffb700 1 -- 172.21.15.92:0/1891654327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4011ea0 con 0x7fd8f0004430 2023-06-09T04:35:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd8f9ffb700 1 -- 172.21.15.92:0/1891654327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f400fd40 con 0x7fd8f0004430 2023-06-09T04:35:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9040b2730 con 0x7fd8f0004430 2023-06-09T04:35:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.117+0000 7fd8f9ffb700 1 -- 172.21.15.92:0/1891654327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4014640 con 0x7fd8f0004430 2023-06-09T04:35:47.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.118+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9040b3470 con 0x7fd8f0004430 2023-06-09T04:35:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.119+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9040b2d80 con 0x7fd8f0004430 2023-06-09T04:35:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.119+0000 7fd8f9ffb700 1 -- 172.21.15.92:0/1891654327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8f40147a0 con 0x7fd8f0004430 2023-06-09T04:35:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.119+0000 7fd8f9ffb700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd8e806ace0 0x7fd8e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.119+0000 7fd8f9ffb700 1 -- 172.21.15.92:0/1891654327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd8f408a030 con 0x7fd8f0004430 2023-06-09T04:35:47.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.120+0000 7fd908a04700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd8e806ace0 0x7fd8e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.120+0000 7fd908a04700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd8e806ace0 0x7fd8e806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd900008fd0 tx=0x7fd90000a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.124+0000 7fd8f9ffb700 1 -- 172.21.15.92:0/1891654327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8f4055050 con 0x7fd8f0004430 2023-06-09T04:35:47.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.312+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd90404e8e0 con 0x7fd8f0004430 2023-06-09T04:35:47.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.313+0000 7fd8f9ffb700 1 -- 172.21.15.92:0/1891654327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8f4058670 con 0x7fd8f0004430 2023-06-09T04:35:47.314 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:47.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd8e806ace0 msgr2=0x7fd8e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd8e806ace0 0x7fd8e806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd900008fd0 tx=0x7fd90000a000 comp rx=0 tx=0).stop 2023-06-09T04:35:47.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0004430 msgr2=0x7fd9040b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0004430 0x7fd9040b2f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4011ca0 tx=0x7fd8f400f520 comp rx=0 tx=0).stop 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 shutdown_connections 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd8e806ace0 0x7fd8e806d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9040c8860 0x7fd904069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8f0004430 0x7fd9040b2f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 --2- 172.21.15.92:0/1891654327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd904150b10 0x7fd9040c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 >> 172.21.15.92:0/1891654327 conn(0x7fd9040cc380 msgr2=0x7fd9040b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:47.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 shutdown_connections 2023-06-09T04:35:47.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.315+0000 7fd90b469700 1 -- 172.21.15.92:0/1891654327 wait complete. 2023-06-09T04:35:47.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.449+0000 7f753f9c8700 1 Processor -- start 2023-06-09T04:35:47.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.449+0000 7f753f9c8700 1 -- start start 2023-06-09T04:35:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753f9c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380ce3d0 0x7f75380713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753f9c8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75380aed50 0x7f75380ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753f9c8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f753815d470 0x7f753815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753f9c8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7538072350 con 0x7f753815d470 2023-06-09T04:35:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753f9c8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75380ce190 con 0x7f75380aed50 2023-06-09T04:35:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753f9c8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75380cece0 con 0x7f75380ce3d0 2023-06-09T04:35:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753df65700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f753815d470 0x7f753815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753df65700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f753815d470 0x7f753815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38642/0 (socket says 172.21.15.92:38642) 2023-06-09T04:35:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.450+0000 7f753df65700 1 -- 172.21.15.92:0/1814952083 learned_addr learned my addr 172.21.15.92:0/1814952083 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753df65700 1 -- 172.21.15.92:0/1814952083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380ce3d0 msgr2=0x7f75380713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753df65700 1 --2- 172.21.15.92:0/1814952083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380ce3d0 0x7f75380713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753df65700 1 -- 172.21.15.92:0/1814952083 >> v2:172.21.15.123:3300/0 conn(0x7f75380aed50 msgr2=0x7f75380ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753df65700 1 --2- 172.21.15.92:0/1814952083 >> v2:172.21.15.123:3300/0 conn(0x7f75380aed50 0x7f75380ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753df65700 1 -- 172.21.15.92:0/1814952083 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7538071ab0 con 0x7f753815d470 2023-06-09T04:35:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753df65700 1 --2- 172.21.15.92:0/1814952083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f753815d8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f75380710a0 tx=0x7f753000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd23427f773beaad server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f752ffff700 1 -- 172.21.15.92:0/1814952083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530010070 con 0x7f753815d470 2023-06-09T04:35:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f752ffff700 1 -- 172.21.15.92:0/1814952083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753000dd90 con 0x7f753815d470 2023-06-09T04:35:47.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f752ffff700 1 -- 172.21.15.92:0/1814952083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753000b310 con 0x7f753815d470 2023-06-09T04:35:47.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 -- 172.21.15.92:0/1814952083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 msgr2=0x7f753815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 --2- 172.21.15.92:0/1814952083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f753815d8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f75380710a0 tx=0x7f753000d430 comp rx=0 tx=0).stop 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 -- 172.21.15.92:0/1814952083 shutdown_connections 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 --2- 172.21.15.92:0/1814952083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f753815d8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 --2- 172.21.15.92:0/1814952083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380ce3d0 0x7f75380713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 --2- 172.21.15.92:0/1814952083 >> v2:172.21.15.123:3300/0 conn(0x7f75380aed50 0x7f75380ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 -- 172.21.15.92:0/1814952083 >> 172.21.15.92:0/1814952083 conn(0x7f75380cc380 msgr2=0x7f75380cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 -- 172.21.15.92:0/1814952083 shutdown_connections 2023-06-09T04:35:47.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.451+0000 7f753f9c8700 1 -- 172.21.15.92:0/1814952083 wait complete. 2023-06-09T04:35:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 Processor -- start 2023-06-09T04:35:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 -- start start 2023-06-09T04:35:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380aed50 0x7f75381f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75380ce3d0 0x7f7538157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f7538157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380c6a70 con 0x7f753815d470 2023-06-09T04:35:47.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380c7ff0 con 0x7f75380ce3d0 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753f9c8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75380c82d0 con 0x7f75380aed50 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753df65700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f7538157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753df65700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f7538157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38648/0 (socket says 172.21.15.92:38648) 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.453+0000 7f753df65700 1 -- 172.21.15.92:0/235539992 learned_addr learned my addr 172.21.15.92:0/235539992 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753d764700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380aed50 0x7f75381f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753cf63700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75380ce3d0 0x7f7538157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753df65700 1 -- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380aed50 msgr2=0x7f75381f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753df65700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380aed50 0x7f75381f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753df65700 1 -- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75380ce3d0 msgr2=0x7f7538157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753df65700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75380ce3d0 0x7f7538157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753df65700 1 -- 172.21.15.92:0/235539992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753000bd40 con 0x7f753815d470 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753df65700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f7538157ea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f753806ff60 tx=0x7f7530004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f752dffb700 1 -- 172.21.15.92:0/235539992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530009a10 con 0x7f753815d470 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f752dffb700 1 -- 172.21.15.92:0/235539992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7530009b90 con 0x7f753815d470 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75380c6d10 con 0x7f753815d470 2023-06-09T04:35:47.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f752dffb700 1 -- 172.21.15.92:0/235539992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530004040 con 0x7f753815d470 2023-06-09T04:35:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.454+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75381f20c0 con 0x7f753815d470 2023-06-09T04:35:47.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.455+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753804e8e0 con 0x7f753815d470 2023-06-09T04:35:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.456+0000 7f752dffb700 1 -- 172.21.15.92:0/235539992 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f75300041a0 con 0x7f753815d470 2023-06-09T04:35:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.456+0000 7f752dffb700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f751c06ace0 0x7f751c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.456+0000 7f752dffb700 1 -- 172.21.15.92:0/235539992 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f753008af90 con 0x7f753815d470 2023-06-09T04:35:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.456+0000 7f753d764700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f751c06ace0 0x7f751c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.457+0000 7f753d764700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f751c06ace0 0x7f751c06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7534006fd0 tx=0x7f7534008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.461+0000 7f752dffb700 1 -- 172.21.15.92:0/235539992 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f75300bd310 con 0x7f753815d470 2023-06-09T04:35:47.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.646+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f75380c9dc0 con 0x7f753815d470 2023-06-09T04:35:47.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.646+0000 7f752dffb700 1 -- 172.21.15.92:0/235539992 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75300560d0 con 0x7f753815d470 2023-06-09T04:35:47.648 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f751c06ace0 msgr2=0x7f751c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f751c06ace0 0x7f751c06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7534006fd0 tx=0x7f7534008040 comp rx=0 tx=0).stop 2023-06-09T04:35:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 msgr2=0x7f7538157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f7538157ea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f753806ff60 tx=0x7f7530004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 shutdown_connections 2023-06-09T04:35:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f751c06ace0 0x7f751c06d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753815d470 0x7f7538157ea0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75380aed50 0x7f75381f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 --2- 172.21.15.92:0/235539992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75380ce3d0 0x7f7538157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.648+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 >> 172.21.15.92:0/235539992 conn(0x7f75380cc380 msgr2=0x7f753806ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.649+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 shutdown_connections 2023-06-09T04:35:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.649+0000 7f753f9c8700 1 -- 172.21.15.92:0/235539992 wait complete. 2023-06-09T04:35:47.671 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-06-09T04:35:47.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-06-09T04:35:47.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.792+0000 7faceb70d700 1 Processor -- start 2023-06-09T04:35:47.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.792+0000 7faceb70d700 1 -- start start 2023-06-09T04:35:47.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7faceb70d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7face40b2070 0x7face40ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7faceb70d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 0x7face40cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7faceb70d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face40d17c0 con 0x7face40cfa70 2023-06-09T04:35:47.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7faceb70d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face40cfee0 con 0x7face40b2070 2023-06-09T04:35:47.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7faceb70d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face40d0720 con 0x7face40cd1a0 2023-06-09T04:35:47.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face94a9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7face40b2070 0x7face40ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face9caa700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7face40cfa70 0x7face40d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36988/0 (socket says 172.21.15.92:36988) 2023-06-09T04:35:47.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face9caa700 1 -- 172.21.15.92:0/2169491908 learned_addr learned my addr 172.21.15.92:0/2169491908 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:47.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face8ca8700 1 --2- 172.21.15.92:0/2169491908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 0x7face40cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face94a9700 1 -- 172.21.15.92:0/2169491908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 msgr2=0x7face40cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:47.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face94a9700 1 --2- 172.21.15.92:0/2169491908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 0x7face40cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face94a9700 1 -- 172.21.15.92:0/2169491908 >> v1:172.21.15.92:6789/0 conn(0x7face40cfa70 legacy=0x7face40d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face94a9700 1 -- 172.21.15.92:0/2169491908 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face40d0a00 con 0x7face40b2070 2023-06-09T04:35:47.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7facdbfff700 1 -- 172.21.15.92:0/2169491908 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532544830 0 0) 0x7facdc001490 con 0x7face40cfa70 2023-06-09T04:35:47.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7face94a9700 1 --2- 172.21.15.92:0/2169491908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40ced00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7face0009840 tx=0x7face0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd328be60e70ab4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.793+0000 7facdbfff700 1 -- 172.21.15.92:0/2169491908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011830 con 0x7face40b2070 2023-06-09T04:35:47.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7facdbfff700 1 -- 172.21.15.92:0/2169491908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face0011990 con 0x7face40b2070 2023-06-09T04:35:47.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7facdbfff700 1 -- 172.21.15.92:0/2169491908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011d20 con 0x7face40b2070 2023-06-09T04:35:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 -- 172.21.15.92:0/2169491908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 msgr2=0x7face40ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 --2- 172.21.15.92:0/2169491908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40ced00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7face0009840 tx=0x7face0009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 -- 172.21.15.92:0/2169491908 shutdown_connections 2023-06-09T04:35:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 --2- 172.21.15.92:0/2169491908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40ced00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 --2- 172.21.15.92:0/2169491908 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 0x7face40cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 -- 172.21.15.92:0/2169491908 >> 172.21.15.92:0/2169491908 conn(0x7face4065a50 msgr2=0x7face4065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 -- 172.21.15.92:0/2169491908 shutdown_connections 2023-06-09T04:35:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.794+0000 7faceb70d700 1 -- 172.21.15.92:0/2169491908 wait complete. 2023-06-09T04:35:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.795+0000 7faceb70d700 1 Processor -- start 2023-06-09T04:35:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.795+0000 7faceb70d700 1 -- start start 2023-06-09T04:35:47.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7faceb70d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7faceb70d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 0x7face40cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7faceb70d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face40cfa70 0x7face41580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7faceb70d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40ca960 con 0x7face40b2070 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7faceb70d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40cac40 con 0x7face40cfa70 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7faceb70d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face415d6b0 con 0x7face40cd1a0 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38658/0 (socket says 172.21.15.92:38658) 2023-06-09T04:35:47.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 -- 172.21.15.92:0/3361634129 learned_addr learned my addr 172.21.15.92:0/3361634129 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 -- 172.21.15.92:0/3361634129 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 msgr2=0x7face40cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 0x7face40cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 -- 172.21.15.92:0/3361634129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face40cfa70 msgr2=0x7face41580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face40cfa70 0x7face41580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.796+0000 7face94a9700 1 -- 172.21.15.92:0/3361634129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face00094a0 con 0x7face40b2070 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.797+0000 7face94a9700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40c9c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7face00090b0 tx=0x7face0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.797+0000 7facd9ffb700 1 -- 172.21.15.92:0/3361634129 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face00038a0 con 0x7face40b2070 2023-06-09T04:35:47.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.797+0000 7facd9ffb700 1 -- 172.21.15.92:0/3361634129 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face001f070 con 0x7face40b2070 2023-06-09T04:35:47.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.797+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face415d990 con 0x7face40b2070 2023-06-09T04:35:47.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.797+0000 7facd9ffb700 1 -- 172.21.15.92:0/3361634129 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0003c20 con 0x7face40b2070 2023-06-09T04:35:47.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.797+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face4158ee0 con 0x7face40b2070 2023-06-09T04:35:47.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.798+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face4159070 con 0x7face40b2070 2023-06-09T04:35:47.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.798+0000 7facd9ffb700 1 -- 172.21.15.92:0/3361634129 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7face001a430 con 0x7face40b2070 2023-06-09T04:35:47.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.799+0000 7facd9ffb700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7facc806ace0 0x7facc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:47.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.799+0000 7facd9ffb700 1 -- 172.21.15.92:0/3361634129 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7face0021070 con 0x7face40b2070 2023-06-09T04:35:47.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.799+0000 7face8ca8700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7facc806ace0 0x7facc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:47.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.799+0000 7face8ca8700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7facc806ace0 0x7facc806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7face41595e0 tx=0x7facd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:47.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:47.803+0000 7facd9ffb700 1 -- 172.21.15.92:0/3361634129 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7face0055fb0 con 0x7face40b2070 2023-06-09T04:35:48.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.067+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7face415deb0 con 0x7face40b2070 2023-06-09T04:35:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.198+0000 7facd9ffb700 1 -- 172.21.15.92:0/3361634129 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v51) v1 ==== 147+0+0 (secure 0 0 0) 0x7face415deb0 con 0x7face40b2070 2023-06-09T04:35:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7facc806ace0 msgr2=0x7facc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7facc806ace0 0x7facc806d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7face41595e0 tx=0x7facd4008040 comp rx=0 tx=0).stop 2023-06-09T04:35:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 msgr2=0x7face40c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40c9c20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7face00090b0 tx=0x7face0004750 comp rx=0 tx=0).stop 2023-06-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 shutdown_connections 2023-06-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7facc806ace0 0x7facc806d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7face40cfa70 0x7face41580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7face40b2070 0x7face40c9c20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 --2- 172.21.15.92:0/3361634129 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7face40cd1a0 0x7face40cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 >> 172.21.15.92:0/3361634129 conn(0x7face4065a50 msgr2=0x7face40b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 shutdown_connections 2023-06-09T04:35:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.201+0000 7faceb70d700 1 -- 172.21.15.92:0/3361634129 wait complete. 2023-06-09T04:35:48.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-06-09T04:35:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.335+0000 7f6aaafcd700 1 Processor -- start 2023-06-09T04:35:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.336+0000 7f6aaafcd700 1 -- start start 2023-06-09T04:35:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.336+0000 7f6aaafcd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6aa40aee60 0x7f6aa40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.336+0000 7f6aaafcd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6aa406a4c0 0x7f6aa406ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.336+0000 7f6aaafcd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa4071dc0 con 0x7f6aa40aea90 2023-06-09T04:35:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.336+0000 7f6aaafcd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa415edf0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.336+0000 7f6aaafcd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa40cf030 con 0x7f6aa40aee60 2023-06-09T04:35:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa956a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6aa406a4c0 0x7f6aa406ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa8d69700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6aa40aea90 0x7f6aa4069db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36998/0 (socket says 172.21.15.92:36998) 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa8d69700 1 -- 172.21.15.92:0/1799931690 learned_addr learned my addr 172.21.15.92:0/1799931690 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa3fff700 1 --2- 172.21.15.92:0/1799931690 >> v2:172.21.15.123:3300/0 conn(0x7f6aa40aee60 0x7f6aa40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa37fe700 1 -- 172.21.15.92:0/1799931690 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161003164 0 0) 0x7f6aa4071dc0 con 0x7f6aa40aea90 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa37fe700 1 -- 172.21.15.92:0/1799931690 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a940035b0 con 0x7f6aa40aea90 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa956a700 1 -- 172.21.15.92:0/1799931690 >> v2:172.21.15.123:3300/0 conn(0x7f6aa40aee60 msgr2=0x7f6aa40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa956a700 1 --2- 172.21.15.92:0/1799931690 >> v2:172.21.15.123:3300/0 conn(0x7f6aa40aee60 0x7f6aa40cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa956a700 1 -- 172.21.15.92:0/1799931690 >> v1:172.21.15.92:6789/0 conn(0x7f6aa40aea90 legacy=0x7f6aa4069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa956a700 1 -- 172.21.15.92:0/1799931690 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa40cf2d0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa956a700 1 --2- 172.21.15.92:0/1799931690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa406ee20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4070a30 tx=0x7f6a9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b15209ba3fea6a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa37fe700 1 -- 172.21.15.92:0/1799931690 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00e070 con 0x7f6aa406a4c0 2023-06-09T04:35:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa37fe700 1 -- 172.21.15.92:0/1799931690 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9c00bb20 con 0x7f6aa406a4c0 2023-06-09T04:35:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.337+0000 7f6aa37fe700 1 -- 172.21.15.92:0/1799931690 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c012430 con 0x7f6aa406a4c0 2023-06-09T04:35:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 -- 172.21.15.92:0/1799931690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 msgr2=0x7f6aa406ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/1799931690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa406ee20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4070a30 tx=0x7f6a9c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 -- 172.21.15.92:0/1799931690 shutdown_connections 2023-06-09T04:35:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/1799931690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa406ee20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/1799931690 >> v2:172.21.15.123:3300/0 conn(0x7f6aa40aee60 0x7f6aa40cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 -- 172.21.15.92:0/1799931690 >> 172.21.15.92:0/1799931690 conn(0x7f6aa40cc380 msgr2=0x7f6aa40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 -- 172.21.15.92:0/1799931690 shutdown_connections 2023-06-09T04:35:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.338+0000 7f6aaafcd700 1 -- 172.21.15.92:0/1799931690 wait complete. 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.339+0000 7f6aaafcd700 1 Processor -- start 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.339+0000 7f6aaafcd700 1 -- start start 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aaafcd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa414ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aaafcd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa40aea90 0x7f6aa40b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aaafcd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa40b13d0 0x7f6aa40c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aaafcd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40b1840 con 0x7f6aa406a4c0 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aaafcd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40c9770 con 0x7f6aa40b13d0 2023-06-09T04:35:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aaafcd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40b1a30 con 0x7f6aa40aea90 2023-06-09T04:35:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa414ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa414ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38672/0 (socket says 172.21.15.92:38672) 2023-06-09T04:35:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 -- 172.21.15.92:0/2614391342 learned_addr learned my addr 172.21.15.92:0/2614391342 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa956a700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa40b13d0 0x7f6aa40c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa3fff700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa40aea90 0x7f6aa40b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 -- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa40aea90 msgr2=0x7f6aa40b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa40aea90 0x7f6aa40b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 -- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa40b13d0 msgr2=0x7f6aa40c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa40b13d0 0x7f6aa40c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa8d69700 1 -- 172.21.15.92:0/2614391342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c009a70 con 0x7f6aa406a4c0 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa956a700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa40b13d0 0x7f6aa40c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.340+0000 7f6aa3fff700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa40aea90 0x7f6aa40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.341+0000 7f6aa8d69700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa414ed20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c700 tx=0x7f6a9800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.341+0000 7f6aa17fa700 1 -- 172.21.15.92:0/2614391342 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98005d70 con 0x7f6aa406a4c0 2023-06-09T04:35:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.341+0000 7f6aa17fa700 1 -- 172.21.15.92:0/2614391342 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a980070d0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.341+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa40b1d10 con 0x7f6aa406a4c0 2023-06-09T04:35:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.341+0000 7f6aa17fa700 1 -- 172.21.15.92:0/2614391342 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a98007460 con 0x7f6aa406a4c0 2023-06-09T04:35:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.341+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa40c4760 con 0x7f6aa406a4c0 2023-06-09T04:35:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.342+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa404e8e0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.342+0000 7f6aa17fa700 1 -- 172.21.15.92:0/2614391342 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a980075c0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.343+0000 7f6aa17fa700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a8806ace0 0x7f6a8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.343+0000 7f6aa17fa700 1 -- 172.21.15.92:0/2614391342 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a98088e90 con 0x7f6aa406a4c0 2023-06-09T04:35:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.343+0000 7f6aa3fff700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a8806ace0 0x7f6a8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.343+0000 7f6aa3fff700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a8806ace0 0x7f6a8806d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6aa415dd20 tx=0x7f6a90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.347+0000 7f6aa17fa700 1 -- 172.21.15.92:0/2614391342 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6a98053fd0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.534+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6aa406a9b0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.537+0000 7f6aa17fa700 1 -- 172.21.15.92:0/2614391342 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a980575f0 con 0x7f6aa406a4c0 2023-06-09T04:35:48.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a8806ace0 msgr2=0x7f6a8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a8806ace0 0x7f6a8806d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6aa415dd20 tx=0x7f6a90008040 comp rx=0 tx=0).stop 2023-06-09T04:35:48.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 msgr2=0x7f6aa414ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa414ed20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800c700 tx=0x7f6a9800cac0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 shutdown_connections 2023-06-09T04:35:48.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6a8806ace0 0x7f6a8806d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6aa40b13d0 0x7f6aa40c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6aa406a4c0 0x7f6aa414ed20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 --2- 172.21.15.92:0/2614391342 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6aa40aea90 0x7f6aa40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 >> 172.21.15.92:0/2614391342 conn(0x7f6aa40cc380 msgr2=0x7f6aa40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:48.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 shutdown_connections 2023-06-09T04:35:48.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.539+0000 7f6aaafcd700 1 -- 172.21.15.92:0/2614391342 wait complete. 2023-06-09T04:35:48.552 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-06-09T04:35:48.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-09T04:35:48.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-09T04:35:48.642 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-06-09T04:35:48.643 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:48.643 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:48.660 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:48.661 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:48.718 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:48.718 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:48.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:48.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.934+0000 7f2a54748700 1 Processor -- start 2023-06-09T04:35:48.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.934+0000 7f2a54748700 1 -- start start 2023-06-09T04:35:48.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a54748700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2a4c0cf010 0x7f2a4c0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a54748700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0ac400 0x7f2a4c06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a54748700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c0ce730 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a54748700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4c06aaf0 con 0x7f2a4c0cf010 2023-06-09T04:35:48.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a54748700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c15cba0 con 0x7f2a4c0ac400 2023-06-09T04:35:48.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a51ce3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2a4c0cf010 0x7f2a4c0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a52ce5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0ac400 0x7f2a4c06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a52ce5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0ac400 0x7f2a4c06e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:47594/0 (socket says 172.21.15.92:47594) 2023-06-09T04:35:48.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a52ce5700 1 -- 172.21.15.92:0/887491574 learned_addr learned my addr 172.21.15.92:0/887491574 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.935+0000 7f2a524e4700 1 --1- 172.21.15.92:0/887491574 >> v1:172.21.15.123:6789/0 conn(0x7f2a4c15c7d0 0x7f2a4c0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59508/0 (socket says 172.21.15.92:59508) 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a52ce5700 1 -- 172.21.15.92:0/887491574 >> v2:172.21.15.123:3300/0 conn(0x7f2a4c0cf010 msgr2=0x7f2a4c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a52ce5700 1 --2- 172.21.15.92:0/887491574 >> v2:172.21.15.123:3300/0 conn(0x7f2a4c0cf010 0x7f2a4c0ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a52ce5700 1 -- 172.21.15.92:0/887491574 >> v1:172.21.15.123:6789/0 conn(0x7f2a4c15c7d0 legacy=0x7f2a4c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a52ce5700 1 -- 172.21.15.92:0/887491574 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c069ee0 con 0x7f2a4c0ac400 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a51ce3700 1 --2- 172.21.15.92:0/887491574 >> v2:172.21.15.123:3300/0 conn(0x7f2a4c0cf010 0x7f2a4c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a52ce5700 1 --2- 172.21.15.92:0/887491574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0ac400 0x7f2a4c06e220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009e50 tx=0x7f2a4400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d52e67ee5e9c819a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:48.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a514e2700 1 -- 172.21.15.92:0/887491574 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400e070 con 0x7f2a4c0ac400 2023-06-09T04:35:48.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a514e2700 1 -- 172.21.15.92:0/887491574 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4400b910 con 0x7f2a4c0ac400 2023-06-09T04:35:48.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a514e2700 1 -- 172.21.15.92:0/887491574 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400bcc0 con 0x7f2a4c0ac400 2023-06-09T04:35:48.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a54748700 1 -- 172.21.15.92:0/887491574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0ac400 msgr2=0x7f2a4c06e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.936+0000 7f2a54748700 1 --2- 172.21.15.92:0/887491574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0ac400 0x7f2a4c06e220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009e50 tx=0x7f2a4400b330 comp rx=0 tx=0).stop 2023-06-09T04:35:48.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.937+0000 7f2a54748700 1 -- 172.21.15.92:0/887491574 shutdown_connections 2023-06-09T04:35:48.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.937+0000 7f2a54748700 1 --2- 172.21.15.92:0/887491574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0ac400 0x7f2a4c06e220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.937+0000 7f2a54748700 1 --2- 172.21.15.92:0/887491574 >> v2:172.21.15.123:3300/0 conn(0x7f2a4c0cf010 0x7f2a4c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.937+0000 7f2a54748700 1 -- 172.21.15.92:0/887491574 >> 172.21.15.92:0/887491574 conn(0x7f2a4c098c10 msgr2=0x7f2a4c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:48.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.937+0000 7f2a54748700 1 -- 172.21.15.92:0/887491574 shutdown_connections 2023-06-09T04:35:48.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.937+0000 7f2a54748700 1 -- 172.21.15.92:0/887491574 wait complete. 2023-06-09T04:35:48.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 Processor -- start 2023-06-09T04:35:48.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 -- start start 2023-06-09T04:35:48.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c0ac400 0x7f2a4c0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0cf010 0x7f2a4c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a4c15c7d0 0x7f2a4c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b57a0 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0c46d0 con 0x7f2a4c0ac400 2023-06-09T04:35:48.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.938+0000 7f2a54748700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0c4c20 con 0x7f2a4c0cf010 2023-06-09T04:35:48.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a4c15c7d0 0x7f2a4c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a4c15c7d0 0x7f2a4c0c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38690/0 (socket says 172.21.15.92:38690) 2023-06-09T04:35:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 -- 172.21.15.92:0/4117154100 learned_addr learned my addr 172.21.15.92:0/4117154100 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a524e4700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c0ac400 0x7f2a4c0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a51ce3700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0cf010 0x7f2a4c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 -- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0cf010 msgr2=0x7f2a4c1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0cf010 0x7f2a4c1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 -- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c0ac400 msgr2=0x7f2a4c0b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:48.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c0ac400 0x7f2a4c0b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:48.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 -- 172.21.15.92:0/4117154100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480094a0 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a52ce5700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a4c15c7d0 0x7f2a4c0c9e80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c15cce0 tx=0x7f2a44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:48.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a42ffd700 1 -- 172.21.15.92:0/4117154100 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440092e0 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a42ffd700 1 -- 172.21.15.92:0/4117154100 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44004250 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a42ffd700 1 -- 172.21.15.92:0/4117154100 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44004600 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a44009b30 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.939+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c0c4940 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.940+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c0ae000 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.941+0000 7f2a42ffd700 1 -- 172.21.15.92:0/4117154100 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a44004760 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.941+0000 7f2a42ffd700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2a3006ace0 0x7f2a3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.941+0000 7f2a42ffd700 1 -- 172.21.15.92:0/4117154100 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a4408a9d0 con 0x7f2a4c15c7d0 2023-06-09T04:35:48.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.941+0000 7f2a524e4700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2a3006ace0 0x7f2a3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.941+0000 7f2a524e4700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2a3006ace0 0x7f2a3006d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000fb0 tx=0x7f2a3c007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:48.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:48.945+0000 7f2a42ffd700 1 -- 172.21.15.92:0/4117154100 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a44055b40 con 0x7f2a4c15c7d0 2023-06-09T04:35:49.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.138+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a4c0740e0 con 0x7f2a4c15c7d0 2023-06-09T04:35:49.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.139+0000 7f2a42ffd700 1 -- 172.21.15.92:0/4117154100 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f2a44059160 con 0x7f2a4c15c7d0 2023-06-09T04:35:49.141 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:49.141 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2a3006ace0 msgr2=0x7f2a3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2a3006ace0 0x7f2a3006d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000fb0 tx=0x7f2a3c007ec0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a4c15c7d0 msgr2=0x7f2a4c0c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a4c15c7d0 0x7f2a4c0c9e80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c15cce0 tx=0x7f2a44004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 shutdown_connections 2023-06-09T04:35:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2a3006ace0 0x7f2a3006d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a4c15c7d0 0x7f2a4c0c9e80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c0ac400 0x7f2a4c0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 --2- 172.21.15.92:0/4117154100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c0cf010 0x7f2a4c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 >> 172.21.15.92:0/4117154100 conn(0x7f2a4c098c10 msgr2=0x7f2a4c0ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:49.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 shutdown_connections 2023-06-09T04:35:49.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.141+0000 7f2a54748700 1 -- 172.21.15.92:0/4117154100 wait complete. 2023-06-09T04:35:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:49.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.279+0000 7fc55b0b9700 1 Processor -- start 2023-06-09T04:35:49.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.279+0000 7fc55b0b9700 1 -- start start 2023-06-09T04:35:49.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.279+0000 7fc55b0b9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc554069db0 0x7fc55415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.279+0000 7fc55b0b9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cf440 0x7fc55406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.279+0000 7fc55b0b9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55415d040 con 0x7fc5540cce60 2023-06-09T04:35:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.279+0000 7fc55b0b9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc554160ab0 con 0x7fc554069db0 2023-06-09T04:35:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.279+0000 7fc55b0b9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55415ca10 con 0x7fc5540cf440 2023-06-09T04:35:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc558e55700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc554069db0 0x7fc55415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc559656700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc5540cce60 0x7fc55406ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37030/0 (socket says 172.21.15.92:37030) 2023-06-09T04:35:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc559656700 1 -- 172.21.15.92:0/3672029460 learned_addr learned my addr 172.21.15.92:0/3672029460 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc553fff700 1 --2- 172.21.15.92:0/3672029460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cf440 0x7fc55406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc558e55700 1 -- 172.21.15.92:0/3672029460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cf440 msgr2=0x7fc55406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc558e55700 1 --2- 172.21.15.92:0/3672029460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cf440 0x7fc55406a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc558e55700 1 -- 172.21.15.92:0/3672029460 >> v1:172.21.15.92:6789/0 conn(0x7fc5540cce60 legacy=0x7fc55406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc558e55700 1 -- 172.21.15.92:0/3672029460 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55406a6c0 con 0x7fc554069db0 2023-06-09T04:35:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc5537fe700 1 -- 172.21.15.92:0/3672029460 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92108626 0 0) 0x7fc55415d040 con 0x7fc5540cce60 2023-06-09T04:35:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc558e55700 1 --2- 172.21.15.92:0/3672029460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc55415d4d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc548009840 tx=0x7fc548009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a688704cf4ea9545 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc5537fe700 1 -- 172.21.15.92:0/3672029460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548011830 con 0x7fc554069db0 2023-06-09T04:35:49.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc5537fe700 1 -- 172.21.15.92:0/3672029460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548011990 con 0x7fc554069db0 2023-06-09T04:35:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.280+0000 7fc5537fe700 1 -- 172.21.15.92:0/3672029460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548011d20 con 0x7fc554069db0 2023-06-09T04:35:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 -- 172.21.15.92:0/3672029460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 msgr2=0x7fc55415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/3672029460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc55415d4d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc548009840 tx=0x7fc548009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 -- 172.21.15.92:0/3672029460 shutdown_connections 2023-06-09T04:35:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/3672029460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc55415d4d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/3672029460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cf440 0x7fc55406a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 -- 172.21.15.92:0/3672029460 >> 172.21.15.92:0/3672029460 conn(0x7fc5540cc380 msgr2=0x7fc5540cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 -- 172.21.15.92:0/3672029460 shutdown_connections 2023-06-09T04:35:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.281+0000 7fc55b0b9700 1 -- 172.21.15.92:0/3672029460 wait complete. 2023-06-09T04:35:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.282+0000 7fc55b0b9700 1 Processor -- start 2023-06-09T04:35:49.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.282+0000 7fc55b0b9700 1 -- start start 2023-06-09T04:35:49.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc55b0b9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc5540c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc55b0b9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cce60 0x7fc5540af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc55b0b9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5540cf440 0x7fc5541e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc55b0b9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540b56c0 con 0x7fc554069db0 2023-06-09T04:35:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc55b0b9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540b5100 con 0x7fc5540cf440 2023-06-09T04:35:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc55b0b9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540b53e0 con 0x7fc5540cce60 2023-06-09T04:35:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc5540c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc5540c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38700/0 (socket says 172.21.15.92:38700) 2023-06-09T04:35:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 -- 172.21.15.92:0/1453633087 learned_addr learned my addr 172.21.15.92:0/1453633087 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc553fff700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cce60 0x7fc5540af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc559656700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5540cf440 0x7fc5541e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 -- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cce60 msgr2=0x7fc5540af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cce60 0x7fc5540af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 -- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5540cf440 msgr2=0x7fc5541e7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5540cf440 0x7fc5541e7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 -- 172.21.15.92:0/1453633087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480094a0 con 0x7fc554069db0 2023-06-09T04:35:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.283+0000 7fc558e55700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc5540c8570 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc554070ca0 tx=0x7fc548004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.284+0000 7fc5517fa700 1 -- 172.21.15.92:0/1453633087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480038a0 con 0x7fc554069db0 2023-06-09T04:35:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.284+0000 7fc5517fa700 1 -- 172.21.15.92:0/1453633087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54801f070 con 0x7fc554069db0 2023-06-09T04:35:49.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.284+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5540c6b70 con 0x7fc554069db0 2023-06-09T04:35:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.284+0000 7fc5517fa700 1 -- 172.21.15.92:0/1453633087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548003c40 con 0x7fc554069db0 2023-06-09T04:35:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.284+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5540c8ab0 con 0x7fc554069db0 2023-06-09T04:35:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.285+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5540c7090 con 0x7fc554069db0 2023-06-09T04:35:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.285+0000 7fc5517fa700 1 -- 172.21.15.92:0/1453633087 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc548026480 con 0x7fc554069db0 2023-06-09T04:35:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.286+0000 7fc5517fa700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc53806ad10 0x7fc53806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.286+0000 7fc5517fa700 1 -- 172.21.15.92:0/1453633087 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc548021070 con 0x7fc554069db0 2023-06-09T04:35:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.286+0000 7fc553fff700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc53806ad10 0x7fc53806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.286+0000 7fc553fff700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc53806ad10 0x7fc53806d1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5541568b0 tx=0x7fc540008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.290+0000 7fc5517fa700 1 -- 172.21.15.92:0/1453633087 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc5540c7090 con 0x7fc554069db0 2023-06-09T04:35:49.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.480+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5540c7090 con 0x7fc554069db0 2023-06-09T04:35:49.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.481+0000 7fc5517fa700 1 -- 172.21.15.92:0/1453633087 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5540c7090 con 0x7fc554069db0 2023-06-09T04:35:49.483 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:49.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc53806ad10 msgr2=0x7fc53806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc53806ad10 0x7fc53806d1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5541568b0 tx=0x7fc540008040 comp rx=0 tx=0).stop 2023-06-09T04:35:49.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 msgr2=0x7fc5540c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc5540c8570 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc554070ca0 tx=0x7fc548004750 comp rx=0 tx=0).stop 2023-06-09T04:35:49.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 shutdown_connections 2023-06-09T04:35:49.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc53806ad10 0x7fc53806d1c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5540cf440 0x7fc5541e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc554069db0 0x7fc5540c8570 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 --2- 172.21.15.92:0/1453633087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5540cce60 0x7fc5540af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 >> 172.21.15.92:0/1453633087 conn(0x7fc5540cc380 msgr2=0x7fc554071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:49.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 shutdown_connections 2023-06-09T04:35:49.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.483+0000 7fc55b0b9700 1 -- 172.21.15.92:0/1453633087 wait complete. 2023-06-09T04:35:49.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:49.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.618+0000 7f16f913c700 1 Processor -- start 2023-06-09T04:35:49.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.618+0000 7f16f913c700 1 -- start start 2023-06-09T04:35:49.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f913c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16f40ce3d0 0x7f16f40713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f913c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f40ce190 con 0x7f16f415d470 2023-06-09T04:35:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f913c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f4071ab0 con 0x7f16f40aed50 2023-06-09T04:35:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f913c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f4072350 con 0x7f16f40ce3d0 2023-06-09T04:35:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16f40ce3d0 0x7f16f40713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f16f415d470 0x7f16f41596e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37044/0 (socket says 172.21.15.92:37044) 2023-06-09T04:35:49.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f359e700 1 -- 172.21.15.92:0/291942941 learned_addr learned my addr 172.21.15.92:0/291942941 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16f40ce3d0 0x7f16f40713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38708/0 (socket says 172.21.15.92:38708) 2023-06-09T04:35:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f2d9d700 1 -- 172.21.15.92:0/291942941 >> v1:172.21.15.123:6789/0 conn(0x7f16f40aed50 legacy=0x7f16f40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.619+0000 7f16f2d9d700 1 -- 172.21.15.92:0/291942941 >> v1:172.21.15.92:6789/0 conn(0x7f16f415d470 legacy=0x7f16f41596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f2d9d700 1 -- 172.21.15.92:0/291942941 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4071d90 con 0x7f16f40ce3d0 2023-06-09T04:35:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f1d9b700 1 -- 172.21.15.92:0/291942941 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245510432 0 0) 0x7f16f40ce190 con 0x7f16f415d470 2023-06-09T04:35:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f2d9d700 1 --2- 172.21.15.92:0/291942941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40ce3d0 0x7f16f40713b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009840 tx=0x7f16e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fcefd633dde3d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f1d9b700 1 -- 172.21.15.92:0/291942941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8004230 con 0x7f16f40ce3d0 2023-06-09T04:35:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f1d9b700 1 -- 172.21.15.92:0/291942941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e8004390 con 0x7f16f40ce3d0 2023-06-09T04:35:49.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f1d9b700 1 -- 172.21.15.92:0/291942941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80036a0 con 0x7f16f40ce3d0 2023-06-09T04:35:49.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f913c700 1 -- 172.21.15.92:0/291942941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40ce3d0 msgr2=0x7f16f40713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f913c700 1 --2- 172.21.15.92:0/291942941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40ce3d0 0x7f16f40713b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009840 tx=0x7f16e8009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:49.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f913c700 1 -- 172.21.15.92:0/291942941 shutdown_connections 2023-06-09T04:35:49.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f913c700 1 --2- 172.21.15.92:0/291942941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40ce3d0 0x7f16f40713b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f913c700 1 -- 172.21.15.92:0/291942941 >> 172.21.15.92:0/291942941 conn(0x7f16f40cc380 msgr2=0x7f16f40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:49.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f913c700 1 -- 172.21.15.92:0/291942941 shutdown_connections 2023-06-09T04:35:49.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.620+0000 7f16f913c700 1 -- 172.21.15.92:0/291942941 wait complete. 2023-06-09T04:35:49.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.622+0000 7f16f913c700 1 Processor -- start 2023-06-09T04:35:49.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.622+0000 7f16f913c700 1 -- start start 2023-06-09T04:35:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f913c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40aed50 0x7f16f40b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f913c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f40ce3d0 0x7f16f40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f913c700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f415d470 0x7f16f40cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f913c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b4390 con 0x7f16f40aed50 2023-06-09T04:35:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f913c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b1370 con 0x7f16f40ce3d0 2023-06-09T04:35:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f913c700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b3780 con 0x7f16f415d470 2023-06-09T04:35:49.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40aed50 0x7f16f40b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40aed50 0x7f16f40b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38714/0 (socket says 172.21.15.92:38714) 2023-06-09T04:35:49.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 -- 172.21.15.92:0/518113192 learned_addr learned my addr 172.21.15.92:0/518113192 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:49.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f259c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f40ce3d0 0x7f16f40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 -- 172.21.15.92:0/518113192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f415d470 msgr2=0x7f16f40cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f415d470 0x7f16f40cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 -- 172.21.15.92:0/518113192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f40ce3d0 msgr2=0x7f16f40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f40ce3d0 0x7f16f40cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 -- 172.21.15.92:0/518113192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80094a0 con 0x7f16f40aed50 2023-06-09T04:35:49.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f2d9d700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40aed50 0x7f16f40b0e30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f16e80090b0 tx=0x7f16e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16db7fe700 1 -- 172.21.15.92:0/518113192 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8011630 con 0x7f16f40aed50 2023-06-09T04:35:49.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16db7fe700 1 -- 172.21.15.92:0/518113192 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e801e070 con 0x7f16f40aed50 2023-06-09T04:35:49.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f40b3a60 con 0x7f16f40aed50 2023-06-09T04:35:49.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.623+0000 7f16db7fe700 1 -- 172.21.15.92:0/518113192 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80119b0 con 0x7f16f40aed50 2023-06-09T04:35:49.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.624+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f40b4020 con 0x7f16f40aed50 2023-06-09T04:35:49.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.625+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f40b4720 con 0x7f16f40aed50 2023-06-09T04:35:49.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.625+0000 7f16db7fe700 1 -- 172.21.15.92:0/518113192 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f16e8011b10 con 0x7f16f40aed50 2023-06-09T04:35:49.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.625+0000 7f16db7fe700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f16e006ace0 0x7f16e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.626+0000 7f16f259c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f16e006ace0 0x7f16e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.626+0000 7f16db7fe700 1 -- 172.21.15.92:0/518113192 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16e808a7b0 con 0x7f16f40aed50 2023-06-09T04:35:49.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.626+0000 7f16f259c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f16e006ace0 0x7f16e006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16f40b5060 tx=0x7f16e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.630+0000 7f16db7fe700 1 -- 172.21.15.92:0/518113192 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16e80558f0 con 0x7f16f40aed50 2023-06-09T04:35:49.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.817+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f16f40cecd0 con 0x7f16f40aed50 2023-06-09T04:35:49.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.817+0000 7f16db7fe700 1 -- 172.21.15.92:0/518113192 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f16e8020020 con 0x7f16f40aed50 2023-06-09T04:35:49.819 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f16e006ace0 msgr2=0x7f16e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f16e006ace0 0x7f16e006d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16f40b5060 tx=0x7f16e4008040 comp rx=0 tx=0).stop 2023-06-09T04:35:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40aed50 msgr2=0x7f16f40b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40aed50 0x7f16f40b0e30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f16e80090b0 tx=0x7f16e8004750 comp rx=0 tx=0).stop 2023-06-09T04:35:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 shutdown_connections 2023-06-09T04:35:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f16e006ace0 0x7f16e006d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f415d470 0x7f16f40cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16f40aed50 0x7f16f40b0e30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 --2- 172.21.15.92:0/518113192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16f40ce3d0 0x7f16f40cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.819+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 >> 172.21.15.92:0/518113192 conn(0x7f16f40cc380 msgr2=0x7f16f40cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.820+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 shutdown_connections 2023-06-09T04:35:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.820+0000 7f16f913c700 1 -- 172.21.15.92:0/518113192 wait complete. 2023-06-09T04:35:49.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.954+0000 7f09ab4d0700 1 Processor -- start 2023-06-09T04:35:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.954+0000 7f09ab4d0700 1 -- start start 2023-06-09T04:35:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09ab4d0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09a4070990 0x7f09a40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09ab4d0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09a40cd5e0 0x7f09a40c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09ab4d0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40cda00 con 0x7f09a4156aa0 2023-06-09T04:35:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09ab4d0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40adf10 con 0x7f09a40cd5e0 2023-06-09T04:35:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09ab4d0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a41594c0 con 0x7f09a4070990 2023-06-09T04:35:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09a9a6d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09a40cd5e0 0x7f09a40c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09a926c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09a4156aa0 0x7f09a4070280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37054/0 (socket says 172.21.15.92:37054) 2023-06-09T04:35:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09a926c700 1 -- 172.21.15.92:0/3292000694 learned_addr learned my addr 172.21.15.92:0/3292000694 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09a9a6d700 1 -- 172.21.15.92:0/3292000694 >> v2:172.21.15.123:3300/0 conn(0x7f09a4070990 msgr2=0x7f09a40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09a9a6d700 1 --2- 172.21.15.92:0/3292000694 >> v2:172.21.15.123:3300/0 conn(0x7f09a4070990 0x7f09a40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09a9a6d700 1 -- 172.21.15.92:0/3292000694 >> v1:172.21.15.92:6789/0 conn(0x7f09a4156aa0 legacy=0x7f09a4070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.955+0000 7f09a9a6d700 1 -- 172.21.15.92:0/3292000694 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a41596d0 con 0x7f09a40cd5e0 2023-06-09T04:35:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09a9a6d700 1 --2- 172.21.15.92:0/3292000694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a40cd5e0 0x7f09a40c9750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f09a406e790 tx=0x7f099c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9285565c9b1d4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f099bfff700 1 -- 172.21.15.92:0/3292000694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00e070 con 0x7f09a40cd5e0 2023-06-09T04:35:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f099bfff700 1 -- 172.21.15.92:0/3292000694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c00bbd0 con 0x7f09a40cd5e0 2023-06-09T04:35:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f099bfff700 1 -- 172.21.15.92:0/3292000694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c012430 con 0x7f09a40cd5e0 2023-06-09T04:35:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3292000694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a40cd5e0 msgr2=0x7f09a40c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3292000694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a40cd5e0 0x7f09a40c9750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f09a406e790 tx=0x7f099c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3292000694 shutdown_connections 2023-06-09T04:35:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3292000694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a40cd5e0 0x7f09a40c9750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3292000694 >> v2:172.21.15.123:3300/0 conn(0x7f09a4070990 0x7f09a40cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3292000694 >> 172.21.15.92:0/3292000694 conn(0x7f09a406aa40 msgr2=0x7f09a40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3292000694 shutdown_connections 2023-06-09T04:35:49.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.956+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3292000694 wait complete. 2023-06-09T04:35:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.958+0000 7f09ab4d0700 1 Processor -- start 2023-06-09T04:35:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.958+0000 7f09ab4d0700 1 -- start start 2023-06-09T04:35:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09ab4d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a4070990 0x7f09a406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09ab4d0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a40cd5e0 0x7f09a40b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09ab4d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a4156aa0 0x7f09a4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09ab4d0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40b22c0 con 0x7f09a4156aa0 2023-06-09T04:35:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09ab4d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40b1660 con 0x7f09a4070990 2023-06-09T04:35:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09ab4d0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40b1920 con 0x7f09a40cd5e0 2023-06-09T04:35:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a4156aa0 0x7f09a4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a4156aa0 0x7f09a4157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38732/0 (socket says 172.21.15.92:38732) 2023-06-09T04:35:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 -- 172.21.15.92:0/3269567601 learned_addr learned my addr 172.21.15.92:0/3269567601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a926c700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a4070990 0x7f09a406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 -- 172.21.15.92:0/3269567601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a40cd5e0 msgr2=0x7f09a40b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a40cd5e0 0x7f09a40b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 -- 172.21.15.92:0/3269567601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a4070990 msgr2=0x7f09a406d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a4070990 0x7f09a406d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 -- 172.21.15.92:0/3269567601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c009a70 con 0x7f09a4156aa0 2023-06-09T04:35:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.959+0000 7f09a9a6d700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a4156aa0 0x7f09a4157e80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b370 tx=0x7f099c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.960+0000 7f0999ffb700 1 -- 172.21.15.92:0/3269567601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0098f0 con 0x7f09a4156aa0 2023-06-09T04:35:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.960+0000 7f0999ffb700 1 -- 172.21.15.92:0/3269567601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c020070 con 0x7f09a4156aa0 2023-06-09T04:35:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.960+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a40b1c00 con 0x7f09a4156aa0 2023-06-09T04:35:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.960+0000 7f0999ffb700 1 -- 172.21.15.92:0/3269567601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c004260 con 0x7f09a4156aa0 2023-06-09T04:35:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.960+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a40b2c20 con 0x7f09a4156aa0 2023-06-09T04:35:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.961+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40b2400 con 0x7f09a4156aa0 2023-06-09T04:35:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.961+0000 7f0999ffb700 1 -- 172.21.15.92:0/3269567601 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f099c0043c0 con 0x7f09a4156aa0 2023-06-09T04:35:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.962+0000 7f0999ffb700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f098806ace0 0x7f098806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.962+0000 7f0999ffb700 1 -- 172.21.15.92:0/3269567601 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f099c022070 con 0x7f09a4156aa0 2023-06-09T04:35:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.962+0000 7f09a926c700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f098806ace0 0x7f098806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.962+0000 7f09a926c700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f098806ace0 0x7f098806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0994005910 tx=0x7f0994009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:49.966+0000 7f0999ffb700 1 -- 172.21.15.92:0/3269567601 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f09a40b2400 con 0x7f09a4156aa0 2023-06-09T04:35:50.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.152+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09a40b2400 con 0x7f09a4156aa0 2023-06-09T04:35:50.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.152+0000 7f0999ffb700 1 -- 172.21.15.92:0/3269567601 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09a40b2400 con 0x7f09a4156aa0 2023-06-09T04:35:50.154 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:50.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f098806ace0 msgr2=0x7f098806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f098806ace0 0x7f098806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0994005910 tx=0x7f0994009040 comp rx=0 tx=0).stop 2023-06-09T04:35:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a4156aa0 msgr2=0x7f09a4157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a4156aa0 0x7f09a4157e80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f099c00b370 tx=0x7f099c004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 shutdown_connections 2023-06-09T04:35:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f098806ace0 0x7f098806d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a4156aa0 0x7f09a4157e80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a4070990 0x7f09a406d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 --2- 172.21.15.92:0/3269567601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a40cd5e0 0x7f09a40b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 >> 172.21.15.92:0/3269567601 conn(0x7f09a406aa40 msgr2=0x7f09a40c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 shutdown_connections 2023-06-09T04:35:50.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.154+0000 7f09ab4d0700 1 -- 172.21.15.92:0/3269567601 wait complete. 2023-06-09T04:35:50.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:50.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.289+0000 7fe27f651700 1 Processor -- start 2023-06-09T04:35:50.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.289+0000 7fe27f651700 1 -- start start 2023-06-09T04:35:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.290+0000 7fe27f651700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2780b60d0 0x7fe2780d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.290+0000 7fe27f651700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780cb700 con 0x7fe27815e810 2023-06-09T04:35:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.290+0000 7fe27f651700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780b64a0 con 0x7fe2780cda90 2023-06-09T04:35:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.290+0000 7fe27f651700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2780ccf50 con 0x7fe2780b60d0 2023-06-09T04:35:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.290+0000 7fe27cbec700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe27815e810 0x7fe2780cd380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37068/0 (socket says 172.21.15.92:37068) 2023-06-09T04:35:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.290+0000 7fe27cbec700 1 -- 172.21.15.92:0/1099580649 learned_addr learned my addr 172.21.15.92:0/1099580649 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.290+0000 7fe27d3ed700 1 --2- 172.21.15.92:0/1099580649 >> v2:172.21.15.123:3300/0 conn(0x7fe2780b60d0 0x7fe2780d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe26ffff700 1 -- 172.21.15.92:0/1099580649 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934412405 0 0) 0x7fe274001110 con 0x7fe27815e810 2023-06-09T04:35:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe26ffff700 1 -- 172.21.15.92:0/1099580649 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2640035b0 con 0x7fe27815e810 2023-06-09T04:35:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe27d3ed700 1 -- 172.21.15.92:0/1099580649 >> v1:172.21.15.123:6789/0 conn(0x7fe2780cda90 legacy=0x7fe2780ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe27d3ed700 1 -- 172.21.15.92:0/1099580649 >> v1:172.21.15.92:6789/0 conn(0x7fe27815e810 legacy=0x7fe2780cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe27d3ed700 1 -- 172.21.15.92:0/1099580649 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780cb140 con 0x7fe2780b60d0 2023-06-09T04:35:50.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe27d3ed700 1 --2- 172.21.15.92:0/1099580649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2780b60d0 0x7fe2780d03b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe268009840 tx=0x7fe268009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1842ff03cbdd1db2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe26ffff700 1 -- 172.21.15.92:0/1099580649 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268011830 con 0x7fe2780b60d0 2023-06-09T04:35:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe26ffff700 1 -- 172.21.15.92:0/1099580649 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe268011990 con 0x7fe2780b60d0 2023-06-09T04:35:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.291+0000 7fe26ffff700 1 -- 172.21.15.92:0/1099580649 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268011d20 con 0x7fe2780b60d0 2023-06-09T04:35:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.292+0000 7fe27f651700 1 -- 172.21.15.92:0/1099580649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2780b60d0 msgr2=0x7fe2780d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.292+0000 7fe27f651700 1 --2- 172.21.15.92:0/1099580649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2780b60d0 0x7fe2780d03b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe268009840 tx=0x7fe268009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.292+0000 7fe27f651700 1 -- 172.21.15.92:0/1099580649 shutdown_connections 2023-06-09T04:35:50.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.292+0000 7fe27f651700 1 --2- 172.21.15.92:0/1099580649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2780b60d0 0x7fe2780d03b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.292+0000 7fe27f651700 1 -- 172.21.15.92:0/1099580649 >> 172.21.15.92:0/1099580649 conn(0x7fe27806e4d0 msgr2=0x7fe27806e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.292+0000 7fe27f651700 1 -- 172.21.15.92:0/1099580649 shutdown_connections 2023-06-09T04:35:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.292+0000 7fe27f651700 1 -- 172.21.15.92:0/1099580649 wait complete. 2023-06-09T04:35:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.293+0000 7fe27f651700 1 Processor -- start 2023-06-09T04:35:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.293+0000 7fe27f651700 1 -- start start 2023-06-09T04:35:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27f651700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2780b60d0 0x7fe2781e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27f651700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2780cda90 0x7fe2780c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27f651700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27815e810 0x7fe2780c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27f651700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2781e4050 con 0x7fe2780cda90 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27f651700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2781e4240 con 0x7fe27815e810 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27f651700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780c9bd0 con 0x7fe2780b60d0 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2780cda90 0x7fe2780c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2780cda90 0x7fe2780c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38744/0 (socket says 172.21.15.92:38744) 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 -- 172.21.15.92:0/596638581 learned_addr learned my addr 172.21.15.92:0/596638581 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27d3ed700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2780b60d0 0x7fe2781e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27dbee700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27815e810 0x7fe2780c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 -- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2780b60d0 msgr2=0x7fe2781e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2780b60d0 0x7fe2781e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 -- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27815e810 msgr2=0x7fe2780c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27815e810 0x7fe2780c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 -- 172.21.15.92:0/596638581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680094a0 con 0x7fe2780cda90 2023-06-09T04:35:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27dbee700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27815e810 0x7fe2780c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27cbec700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2780cda90 0x7fe2780c6990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe274007fd0 tx=0x7fe27400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe26dffb700 1 -- 172.21.15.92:0/596638581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274005d70 con 0x7fe2780cda90 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe26dffb700 1 -- 172.21.15.92:0/596638581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2740070d0 con 0x7fe2780cda90 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2780c9e70 con 0x7fe2780cda90 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.294+0000 7fe26dffb700 1 -- 172.21.15.92:0/596638581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274007460 con 0x7fe2780cda90 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.295+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2780c6ed0 con 0x7fe2780cda90 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.296+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2780cae30 con 0x7fe2780cda90 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.296+0000 7fe26dffb700 1 -- 172.21.15.92:0/596638581 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2740075c0 con 0x7fe2780cda90 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.297+0000 7fe26dffb700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe25c06ace0 0x7fe25c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.297+0000 7fe27d3ed700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe25c06ace0 0x7fe25c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.297+0000 7fe26dffb700 1 -- 172.21.15.92:0/596638581 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe2740888a0 con 0x7fe2780cda90 2023-06-09T04:35:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.297+0000 7fe27d3ed700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe25c06ace0 0x7fe25c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe2680090b0 tx=0x7fe26801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.301+0000 7fe26dffb700 1 -- 172.21.15.92:0/596638581 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe274053950 con 0x7fe2780cda90 2023-06-09T04:35:50.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.489+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2780cb700 con 0x7fe2780cda90 2023-06-09T04:35:50.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.489+0000 7fe26dffb700 1 -- 172.21.15.92:0/596638581 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2780cb700 con 0x7fe2780cda90 2023-06-09T04:35:50.491 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe25c06ace0 msgr2=0x7fe25c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe25c06ace0 0x7fe25c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe2680090b0 tx=0x7fe26801c040 comp rx=0 tx=0).stop 2023-06-09T04:35:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2780cda90 msgr2=0x7fe2780c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2780cda90 0x7fe2780c6990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe274007fd0 tx=0x7fe27400ca90 comp rx=0 tx=0).stop 2023-06-09T04:35:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 shutdown_connections 2023-06-09T04:35:50.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe25c06ace0 0x7fe25c06d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe27815e810 0x7fe2780c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2780b60d0 0x7fe2781e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 --2- 172.21.15.92:0/596638581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2780cda90 0x7fe2780c6990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.491+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 >> 172.21.15.92:0/596638581 conn(0x7fe27806e4d0 msgr2=0x7fe2780ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.492+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 shutdown_connections 2023-06-09T04:35:50.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.492+0000 7fe27f651700 1 -- 172.21.15.92:0/596638581 wait complete. 2023-06-09T04:35:50.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-09T04:35:50.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.624+0000 7f0cea7f6700 1 Processor -- start 2023-06-09T04:35:50.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.624+0000 7f0cea7f6700 1 -- start start 2023-06-09T04:35:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0cea7f6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0cea7f6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ce40afef0 0x7f0ce40cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0cea7f6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0ce40af9a0 0x7f0ce406f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0cea7f6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce406eca0 con 0x7f0ce40afef0 2023-06-09T04:35:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0cea7f6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce40d07f0 con 0x7f0ce40af9a0 2023-06-09T04:35:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0cea7f6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40cdd10 con 0x7f0ce415a380 2023-06-09T04:35:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ce40afef0 0x7f0ce40cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ce40afef0 0x7f0ce40cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38752/0 (socket says 172.21.15.92:38752) 2023-06-09T04:35:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 -- 172.21.15.92:0/3832971534 learned_addr learned my addr 172.21.15.92:0/3832971534 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce8d93700 1 --2- 172.21.15.92:0/3832971534 >> v2:172.21.15.123:3300/0 conn(0x7f0ce40af9a0 0x7f0ce406f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce3fff700 1 --2- 172.21.15.92:0/3832971534 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 -- 172.21.15.92:0/3832971534 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 msgr2=0x7f0ce40ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 --2- 172.21.15.92:0/3832971534 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 -- 172.21.15.92:0/3832971534 >> v2:172.21.15.123:3300/0 conn(0x7f0ce40af9a0 msgr2=0x7f0ce406f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 --2- 172.21.15.92:0/3832971534 >> v2:172.21.15.123:3300/0 conn(0x7f0ce40af9a0 0x7f0ce406f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce37fe700 1 -- 172.21.15.92:0/3832971534 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce415a750 con 0x7f0ce40afef0 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.625+0000 7f0ce8d93700 1 --2- 172.21.15.92:0/3832971534 >> v2:172.21.15.123:3300/0 conn(0x7f0ce40af9a0 0x7f0ce406f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0ce37fe700 1 --2- 172.21.15.92:0/3832971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce40cea80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ce406e3f0 tx=0x7f0ccc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a8496a48387d52 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:50.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0ce2ffd700 1 -- 172.21.15.92:0/3832971534 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00b860 con 0x7f0ce40afef0 2023-06-09T04:35:50.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0ce2ffd700 1 -- 172.21.15.92:0/3832971534 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc00b9c0 con 0x7f0ce40afef0 2023-06-09T04:35:50.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0ce2ffd700 1 -- 172.21.15.92:0/3832971534 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00bd50 con 0x7f0ce40afef0 2023-06-09T04:35:50.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 -- 172.21.15.92:0/3832971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 msgr2=0x7f0ce40cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/3832971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce40cea80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ce406e3f0 tx=0x7f0ccc009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:50.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 -- 172.21.15.92:0/3832971534 shutdown_connections 2023-06-09T04:35:50.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/3832971534 >> v2:172.21.15.123:3300/0 conn(0x7f0ce40af9a0 0x7f0ce406f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/3832971534 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/3832971534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce40cea80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 -- 172.21.15.92:0/3832971534 >> 172.21.15.92:0/3832971534 conn(0x7f0ce4089fe0 msgr2=0x7f0ce40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 -- 172.21.15.92:0/3832971534 shutdown_connections 2023-06-09T04:35:50.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.626+0000 7f0cea7f6700 1 -- 172.21.15.92:0/3832971534 wait complete. 2023-06-09T04:35:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.627+0000 7f0cea7f6700 1 Processor -- start 2023-06-09T04:35:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0cea7f6700 1 -- start start 2023-06-09T04:35:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0cea7f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ce40af9a0 0x7f0ce4149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0cea7f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce4151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0cea7f6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0cea7f6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4151860 con 0x7f0ce40afef0 2023-06-09T04:35:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0cea7f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40b48d0 con 0x7f0ce40af9a0 2023-06-09T04:35:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0cea7f6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40b45f0 con 0x7f0ce415a380 2023-06-09T04:35:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0ce37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce4151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0ce37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce4151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38756/0 (socket says 172.21.15.92:38756) 2023-06-09T04:35:50.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.628+0000 7f0ce37fe700 1 -- 172.21.15.92:0/2400018059 learned_addr learned my addr 172.21.15.92:0/2400018059 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce3fff700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ce40af9a0 0x7f0ce4149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce8d93700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce37fe700 1 -- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 msgr2=0x7f0ce40b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce37fe700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce37fe700 1 -- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ce40af9a0 msgr2=0x7f0ce4149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce37fe700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ce40af9a0 0x7f0ce4149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce37fe700 1 -- 172.21.15.92:0/2400018059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc009550 con 0x7f0ce40afef0 2023-06-09T04:35:50.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce37fe700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce4151cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc014040 tx=0x7f0ccc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce0ff9700 1 -- 172.21.15.92:0/2400018059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc004510 con 0x7f0ce40afef0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce0ff9700 1 -- 172.21.15.92:0/2400018059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ccc026070 con 0x7f0ce40afef0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce4151210 con 0x7f0ce40afef0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0ce0ff9700 1 -- 172.21.15.92:0/2400018059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc0038e0 con 0x7f0ce40afef0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.629+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce414a3e0 con 0x7f0ce40afef0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.630+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce40740e0 con 0x7f0ce40afef0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.631+0000 7f0ce0ff9700 1 -- 172.21.15.92:0/2400018059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ccc003a40 con 0x7f0ce40afef0 2023-06-09T04:35:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.631+0000 7f0ce0ff9700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0cc406ace0 0x7f0cc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:50.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.631+0000 7f0ce0ff9700 1 -- 172.21.15.92:0/2400018059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ccc028070 con 0x7f0ce40afef0 2023-06-09T04:35:50.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.631+0000 7f0ce3fff700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0cc406ace0 0x7f0cc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:50.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.631+0000 7f0ce3fff700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0cc406ace0 0x7f0cc406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006fd0 tx=0x7f0cd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:50.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.635+0000 7f0ce0ff9700 1 -- 172.21.15.92:0/2400018059 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0ccc05e540 con 0x7f0ce40afef0 2023-06-09T04:35:50.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.838+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ce41519a0 con 0x7f0ce40afef0 2023-06-09T04:35:50.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.838+0000 7f0ce0ff9700 1 -- 172.21.15.92:0/2400018059 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0ccc061b60 con 0x7f0ce40afef0 2023-06-09T04:35:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0cc406ace0 msgr2=0x7f0cc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0cc406ace0 0x7f0cc406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006fd0 tx=0x7f0cd4008040 comp rx=0 tx=0).stop 2023-06-09T04:35:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 msgr2=0x7f0ce4151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:50.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce4151cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc014040 tx=0x7f0ccc004750 comp rx=0 tx=0).stop 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 shutdown_connections 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0cc406ace0 0x7f0cc406d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ce415a380 0x7f0ce40b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ce40af9a0 0x7f0ce4149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 --2- 172.21.15.92:0/2400018059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ce40afef0 0x7f0ce4151cf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.840+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 >> 172.21.15.92:0/2400018059 conn(0x7f0ce4089fe0 msgr2=0x7f0ce40cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.841+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 shutdown_connections 2023-06-09T04:35:50.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:50.841+0000 7f0cea7f6700 1 -- 172.21.15.92:0/2400018059 wait complete. 2023-06-09T04:35:50.844 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-06-09T04:35:50.854 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-06-09T04:35:50.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:50.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-06-09T04:35:50.896 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:35:50.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:50.897 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-09T04:35:50.960 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:50.961 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:50.983 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:50.983 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:51.043 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:51.043 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:51.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.222+0000 7f1aebcd7700 1 Processor -- start 2023-06-09T04:35:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.222+0000 7f1aebcd7700 1 -- start start 2023-06-09T04:35:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1aebcd7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ae415a340 0x7f1ae40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1aebcd7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 0x7f1ae40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1aebcd7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae40cc940 con 0x7f1ae4159f70 2023-06-09T04:35:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1aebcd7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae40d07d0 con 0x7f1ae415a340 2023-06-09T04:35:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1aebcd7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae415e6c0 con 0x7f1ae40cb000 2023-06-09T04:35:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1ae9a73700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1ae4159f70 0x7f1ae40cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37096/0 (socket says 172.21.15.92:37096) 2023-06-09T04:35:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1ae9a73700 1 -- 172.21.15.92:0/1904134137 learned_addr learned my addr 172.21.15.92:0/1904134137 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457450789 0 0) 0x7f1ae40cc940 con 0x7f1ae4159f70 2023-06-09T04:35:51.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1acc0035b0 con 0x7f1ae4159f70 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1aea274700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 0x7f1ae40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.223+0000 7f1ae9272700 1 --2- 172.21.15.92:0/1904134137 >> v2:172.21.15.123:3300/0 conn(0x7f1ae415a340 0x7f1ae40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3829236715 0 0) 0x7f1acc0035b0 con 0x7f1ae4159f70 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 msgr2=0x7f1ae40d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 0x7f1ae40d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 >> v2:172.21.15.123:3300/0 conn(0x7f1ae415a340 msgr2=0x7f1ae40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 --2- 172.21.15.92:0/1904134137 >> v2:172.21.15.123:3300/0 conn(0x7f1ae415a340 0x7f1ae40cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae415ebf0 con 0x7f1ae4159f70 2023-06-09T04:35:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f1adc0019f0 con 0x7f1ae4159f70 2023-06-09T04:35:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae9272700 1 --2- 172.21.15.92:0/1904134137 >> v2:172.21.15.123:3300/0 conn(0x7f1ae415a340 0x7f1ae40cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 >> v1:172.21.15.92:6789/0 conn(0x7f1ae4159f70 legacy=0x7f1ae40cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc003a00 0x7f1acc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 0x7f1acc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1acc004d90 0x7f1acc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40d07d0 con 0x7f1acc004d90 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae415e6c0 con 0x7f1acc003a00 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc00baf0 con 0x7f1ae40cb000 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1aea274700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1acc004d90 0x7f1acc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae9a73700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc003a00 0x7f1acc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae9272700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 0x7f1acc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1aea274700 1 -- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 msgr2=0x7f1acc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1aea274700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae40cb000 0x7f1acc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1aea274700 1 -- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc003a00 msgr2=0x7f1acc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1aea274700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1acc003a00 0x7f1acc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1aea274700 1 -- 172.21.15.92:0/1904134137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc0035b0 con 0x7f1acc004d90 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1aea274700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1acc004d90 0x7f1acc005200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009860 tx=0x7f1acc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638ed5805e8be55d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400c070 con 0x7f1acc004d90 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad4013610 con 0x7f1acc004d90 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.224+0000 7f1ae8a71700 1 -- 172.21.15.92:0/1904134137 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40139a0 con 0x7f1acc004d90 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.225+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1904134137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1acc004d90 msgr2=0x7f1acc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.225+0000 7f1aebcd7700 1 --2- 172.21.15.92:0/1904134137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1acc004d90 0x7f1acc005200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009860 tx=0x7f1acc005880 comp rx=0 tx=0).stop 2023-06-09T04:35:51.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.225+0000 7f1aea274700 1 -- 172.21.15.92:0/1904134137 reap_dead start 2023-06-09T04:35:51.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.225+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1904134137 shutdown_connections 2023-06-09T04:35:51.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.225+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1904134137 >> 172.21.15.92:0/1904134137 conn(0x7f1ae406e4e0 msgr2=0x7f1ae406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:51.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.225+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1904134137 shutdown_connections 2023-06-09T04:35:51.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.225+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1904134137 wait complete. 2023-06-09T04:35:51.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.226+0000 7f1aebcd7700 1 Processor -- start 2023-06-09T04:35:51.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.226+0000 7f1aebcd7700 1 -- start start 2023-06-09T04:35:51.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aebcd7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae4154680 0x7f1ae40affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aebcd7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae4154a50 0x7f1ae41e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aebcd7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae41e62c0 0x7f1ae40c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aebcd7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae41e48d0 con 0x7f1ae41e62c0 2023-06-09T04:35:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aebcd7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40c84a0 con 0x7f1ae4154680 2023-06-09T04:35:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aebcd7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae40c8780 con 0x7f1ae4154a50 2023-06-09T04:35:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae41e62c0 0x7f1ae40c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae41e62c0 0x7f1ae40c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38780/0 (socket says 172.21.15.92:38780) 2023-06-09T04:35:51.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 -- 172.21.15.92:0/1363848685 learned_addr learned my addr 172.21.15.92:0/1363848685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1ae9272700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae4154a50 0x7f1ae41e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1ae9a73700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae4154680 0x7f1ae40affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 -- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae4154a50 msgr2=0x7f1ae41e5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae4154a50 0x7f1ae41e5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 -- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae4154680 msgr2=0x7f1ae40affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae4154680 0x7f1ae40affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.227+0000 7f1aea274700 1 -- 172.21.15.92:0/1363848685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40c8a60 con 0x7f1ae41e62c0 2023-06-09T04:35:51.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.228+0000 7f1aea274700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae41e62c0 0x7f1ae40c9260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4015000 tx=0x7f1ad40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.228+0000 7f1ae27fc700 1 -- 172.21.15.92:0/1363848685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4004240 con 0x7f1ae41e62c0 2023-06-09T04:35:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.228+0000 7f1ae27fc700 1 -- 172.21.15.92:0/1363848685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad40043a0 con 0x7f1ae41e62c0 2023-06-09T04:35:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.228+0000 7f1ae27fc700 1 -- 172.21.15.92:0/1363848685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40036a0 con 0x7f1ae41e62c0 2023-06-09T04:35:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.228+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae40c8d40 con 0x7f1ae41e62c0 2023-06-09T04:35:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.228+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae40b0510 con 0x7f1ae41e62c0 2023-06-09T04:35:51.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.229+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae40740e0 con 0x7f1ae41e62c0 2023-06-09T04:35:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.229+0000 7f1ae27fc700 1 -- 172.21.15.92:0/1363848685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ad4003800 con 0x7f1ae41e62c0 2023-06-09T04:35:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.230+0000 7f1ae27fc700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1ad006ace0 0x7f1ad006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.230+0000 7f1ae27fc700 1 -- 172.21.15.92:0/1363848685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ad4000e40 con 0x7f1ae41e62c0 2023-06-09T04:35:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.230+0000 7f1ae9a73700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1ad006ace0 0x7f1ad006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.230+0000 7f1ae9a73700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1ad006ace0 0x7f1ad006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1adc004c50 tx=0x7f1adc006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.234+0000 7f1ae27fc700 1 -- 172.21.15.92:0/1363848685 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1ad4055190 con 0x7f1ae41e62c0 2023-06-09T04:35:51.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.423+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ae40ccb40 con 0x7f1ae41e62c0 2023-06-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.424+0000 7f1ae27fc700 1 -- 172.21.15.92:0/1363848685 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f1ad40587b0 con 0x7f1ae41e62c0 2023-06-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:51.426 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:51.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1ad006ace0 msgr2=0x7f1ad006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1ad006ace0 0x7f1ad006d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1adc004c50 tx=0x7f1adc006040 comp rx=0 tx=0).stop 2023-06-09T04:35:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae41e62c0 msgr2=0x7f1ae40c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae41e62c0 0x7f1ae40c9260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4015000 tx=0x7f1ad40047c0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 shutdown_connections 2023-06-09T04:35:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1ad006ace0 0x7f1ad006d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ae41e62c0 0x7f1ae40c9260 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ae4154680 0x7f1ae40affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 --2- 172.21.15.92:0/1363848685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ae4154a50 0x7f1ae41e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 >> 172.21.15.92:0/1363848685 conn(0x7f1ae406e4e0 msgr2=0x7f1ae415ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:51.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 shutdown_connections 2023-06-09T04:35:51.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.426+0000 7f1aebcd7700 1 -- 172.21.15.92:0/1363848685 wait complete. 2023-06-09T04:35:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:51.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:51.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.562+0000 7fb4213b2700 1 Processor -- start 2023-06-09T04:35:51.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.562+0000 7fb4213b2700 1 -- start start 2023-06-09T04:35:51.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.562+0000 7fb4213b2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb41c0aee60 0x7fb41c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.562+0000 7fb4213b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c06a4c0 0x7fb41c06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.562+0000 7fb4213b2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c071dc0 con 0x7fb41c0aea90 2023-06-09T04:35:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.562+0000 7fb4213b2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c15edf0 con 0x7fb41c0aee60 2023-06-09T04:35:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.562+0000 7fb4213b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0cf030 con 0x7fb41c06a4c0 2023-06-09T04:35:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c06a4c0 0x7fb41c06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41a7fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb41c0aee60 0x7fb41c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41affd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb41c0aea90 0x7fb41c069db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59614/0 (socket says 172.21.15.92:59614) 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41affd700 1 -- 172.21.15.92:0/553601491 learned_addr learned my addr 172.21.15.92:0/553601491 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41a7fc700 1 -- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c06a4c0 msgr2=0x7fb41c06ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41a7fc700 1 --2- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c06a4c0 0x7fb41c06ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41a7fc700 1 -- 172.21.15.92:0/553601491 >> v1:172.21.15.123:6789/0 conn(0x7fb41c0aea90 legacy=0x7fb41c069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41a7fc700 1 -- 172.21.15.92:0/553601491 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0cf2d0 con 0x7fb41c0aee60 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb419ffb700 1 -- 172.21.15.92:0/553601491 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562917990 0 0) 0x7fb41c071dc0 con 0x7fb41c0aea90 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41b7fe700 1 --2- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c06a4c0 0x7fb41c06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:51.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb41a7fc700 1 --2- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aee60 0x7fb41c0cce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009840 tx=0x7fb40c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ad4d123c8b24f3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb419ffb700 1 -- 172.21.15.92:0/553601491 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c011830 con 0x7fb41c0aee60 2023-06-09T04:35:51.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb419ffb700 1 -- 172.21.15.92:0/553601491 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c011990 con 0x7fb41c0aee60 2023-06-09T04:35:51.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.563+0000 7fb419ffb700 1 -- 172.21.15.92:0/553601491 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c011d20 con 0x7fb41c0aee60 2023-06-09T04:35:51.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 -- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aee60 msgr2=0x7fb41c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 --2- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aee60 0x7fb41c0cce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009840 tx=0x7fb40c009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:51.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 -- 172.21.15.92:0/553601491 shutdown_connections 2023-06-09T04:35:51.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 --2- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c06a4c0 0x7fb41c06ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 --2- 172.21.15.92:0/553601491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aee60 0x7fb41c0cce60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 -- 172.21.15.92:0/553601491 >> 172.21.15.92:0/553601491 conn(0x7fb41c0cc380 msgr2=0x7fb41c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 -- 172.21.15.92:0/553601491 shutdown_connections 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.564+0000 7fb4213b2700 1 -- 172.21.15.92:0/553601491 wait complete. 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.565+0000 7fb4213b2700 1 Processor -- start 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.565+0000 7fb4213b2700 1 -- start start 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb4213b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41c06a4c0 0x7fb41c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb4213b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aea90 0x7fb41c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb4213b2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c1564f0 0x7fb41c14bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb4213b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c15a970 con 0x7fb41c06a4c0 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb4213b2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c15ac50 con 0x7fb41c0aea90 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb4213b2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c15af30 con 0x7fb41c1564f0 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41c06a4c0 0x7fb41c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41c06a4c0 0x7fb41c15d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38790/0 (socket says 172.21.15.92:38790) 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 -- 172.21.15.92:0/2569848972 learned_addr learned my addr 172.21.15.92:0/2569848972 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41a7fc700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aea90 0x7fb41c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41b7fe700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c1564f0 0x7fb41c14bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 -- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c1564f0 msgr2=0x7fb41c14bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c1564f0 0x7fb41c14bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 -- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aea90 msgr2=0x7fb41c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aea90 0x7fb41c15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.566+0000 7fb41affd700 1 -- 172.21.15.92:0/2569848972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb414009a70 con 0x7fb41c06a4c0 2023-06-09T04:35:51.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.567+0000 7fb41affd700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41c06a4c0 0x7fb41c15d3f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4040011e0 tx=0x7fb40400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.567+0000 7fb4137fe700 1 -- 172.21.15.92:0/2569848972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404004c80 con 0x7fb41c06a4c0 2023-06-09T04:35:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.567+0000 7fb4137fe700 1 -- 172.21.15.92:0/2569848972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404004e00 con 0x7fb41c06a4c0 2023-06-09T04:35:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.567+0000 7fb4137fe700 1 -- 172.21.15.92:0/2569848972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404007310 con 0x7fb41c06a4c0 2023-06-09T04:35:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.567+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c0094a0 con 0x7fb41c06a4c0 2023-06-09T04:35:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.567+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c15b4f0 con 0x7fb41c06a4c0 2023-06-09T04:35:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.568+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c04e8e0 con 0x7fb41c06a4c0 2023-06-09T04:35:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.568+0000 7fb4137fe700 1 -- 172.21.15.92:0/2569848972 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb404007470 con 0x7fb41c06a4c0 2023-06-09T04:35:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.569+0000 7fb4137fe700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb3fc06ace0 0x7fb3fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.569+0000 7fb4137fe700 1 -- 172.21.15.92:0/2569848972 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4040891e0 con 0x7fb41c06a4c0 2023-06-09T04:35:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.569+0000 7fb41a7fc700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb3fc06ace0 0x7fb3fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.569+0000 7fb41a7fc700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb3fc06ace0 0x7fb3fc06d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb41c06ae90 tx=0x7fb40c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.573+0000 7fb4137fe700 1 -- 172.21.15.92:0/2569848972 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb404054320 con 0x7fb41c06a4c0 2023-06-09T04:35:51.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.759+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb41c06a9b0 con 0x7fb41c06a4c0 2023-06-09T04:35:51.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.759+0000 7fb4137fe700 1 -- 172.21.15.92:0/2569848972 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb404057940 con 0x7fb41c06a4c0 2023-06-09T04:35:51.761 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb3fc06ace0 msgr2=0x7fb3fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb3fc06ace0 0x7fb3fc06d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb41c06ae90 tx=0x7fb40c008030 comp rx=0 tx=0).stop 2023-06-09T04:35:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41c06a4c0 msgr2=0x7fb41c15d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41c06a4c0 0x7fb41c15d3f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4040011e0 tx=0x7fb40400cae0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 shutdown_connections 2023-06-09T04:35:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb3fc06ace0 0x7fb3fc06d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb41c1564f0 0x7fb41c14bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb41c06a4c0 0x7fb41c15d3f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 --2- 172.21.15.92:0/2569848972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb41c0aea90 0x7fb41c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 >> 172.21.15.92:0/2569848972 conn(0x7fb41c0cc380 msgr2=0x7fb41c159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 shutdown_connections 2023-06-09T04:35:51.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.761+0000 7fb4213b2700 1 -- 172.21.15.92:0/2569848972 wait complete. 2023-06-09T04:35:51.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.896+0000 7f888be76700 1 Processor -- start 2023-06-09T04:35:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.897+0000 7f888be76700 1 -- start start 2023-06-09T04:35:51.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.897+0000 7f888be76700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f88840cda90 0x7f88840ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.897+0000 7f888be76700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840cb700 con 0x7f88840d03b0 2023-06-09T04:35:51.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.897+0000 7f888be76700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840b64a0 con 0x7f88840b60d0 2023-06-09T04:35:51.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.897+0000 7f888be76700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840ccf50 con 0x7f88840cda90 2023-06-09T04:35:51.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.897+0000 7f8889411700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f88840d03b0 0x7f88840cd380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37132/0 (socket says 172.21.15.92:37132) 2023-06-09T04:35:51.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.897+0000 7f8889411700 1 -- 172.21.15.92:0/2306281833 learned_addr learned my addr 172.21.15.92:0/2306281833 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:51.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587855319 0 0) 0x7f8880001110 con 0x7f88840d03b0 2023-06-09T04:35:51.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88700035b0 con 0x7f88840d03b0 2023-06-09T04:35:51.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f888a413700 1 --2- 172.21.15.92:0/2306281833 >> v2:172.21.15.123:3300/0 conn(0x7f88840cda90 0x7f88840ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 663968958 0 0) 0x7f88700035b0 con 0x7f88840d03b0 2023-06-09T04:35:51.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 >> v2:172.21.15.123:3300/0 conn(0x7f88840cda90 msgr2=0x7f88840ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 --2- 172.21.15.92:0/2306281833 >> v2:172.21.15.123:3300/0 conn(0x7f88840cda90 0x7f88840ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 >> v1:172.21.15.123:6789/0 conn(0x7f88840b60d0 legacy=0x7f888415e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840cb140 con 0x7f88840d03b0 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f8880001800 con 0x7f88840d03b0 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 >> v1:172.21.15.92:6789/0 conn(0x7f88840d03b0 legacy=0x7f88840cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8870003a00 0x7f8870003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870004850 0x7f8870004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870005200 0x7f887000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840ccf50 con 0x7f8870005200 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887000c080 con 0x7f8870004850 2023-06-09T04:35:51.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887000c240 con 0x7f8870003a00 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f888a413700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870005200 0x7f887000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8889411700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870004850 0x7f8870004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f8889c12700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8870003a00 0x7f8870003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f888a413700 1 -- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8870003a00 msgr2=0x7f8870003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f888a413700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8870003a00 0x7f8870003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f888a413700 1 -- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870004850 msgr2=0x7f8870004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f888a413700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870004850 0x7f8870004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.898+0000 7f888a413700 1 -- 172.21.15.92:0/2306281833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700035b0 con 0x7f8870005200 2023-06-09T04:35:51.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888a413700 1 -- 172.21.15.92:0/2306281833 reap_dead start 2023-06-09T04:35:51.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888a413700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870005200 0x7f887000ba10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f887c009f90 tx=0x7f8870005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2135f574b634368a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c00ec30 con 0x7f8870005200 2023-06-09T04:35:51.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c00f040 con 0x7f8870005200 2023-06-09T04:35:51.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f8888c10700 1 -- 172.21.15.92:0/2306281833 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c014480 con 0x7f8870005200 2023-06-09T04:35:51.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888be76700 1 -- 172.21.15.92:0/2306281833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870005200 msgr2=0x7f887000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888be76700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870005200 0x7f887000ba10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f887c009f90 tx=0x7f8870005880 comp rx=0 tx=0).stop 2023-06-09T04:35:51.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888be76700 1 -- 172.21.15.92:0/2306281833 shutdown_connections 2023-06-09T04:35:51.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888be76700 1 --2- 172.21.15.92:0/2306281833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8870005200 0x7f887000ba10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888be76700 1 -- 172.21.15.92:0/2306281833 >> 172.21.15.92:0/2306281833 conn(0x7f888406e4d0 msgr2=0x7f888406e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:51.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888be76700 1 -- 172.21.15.92:0/2306281833 shutdown_connections 2023-06-09T04:35:51.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.899+0000 7f888be76700 1 -- 172.21.15.92:0/2306281833 wait complete. 2023-06-09T04:35:51.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 Processor -- start 2023-06-09T04:35:51.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 -- start start 2023-06-09T04:35:51.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870005200 0x7f88840ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888406bc20 0x7f88840aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8884158ab0 0x7f888406aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840ca240 con 0x7f8884158ab0 2023-06-09T04:35:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888406c880 con 0x7f8870005200 2023-06-09T04:35:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888be76700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888406cb60 con 0x7f888406bc20 2023-06-09T04:35:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888a413700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8884158ab0 0x7f888406aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888a413700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8884158ab0 0x7f888406aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38814/0 (socket says 172.21.15.92:38814) 2023-06-09T04:35:51.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f888a413700 1 -- 172.21.15.92:0/2115928890 learned_addr learned my addr 172.21.15.92:0/2115928890 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:51.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f8889c12700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870005200 0x7f88840ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.901+0000 7f8889411700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888406bc20 0x7f88840aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888a413700 1 -- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888406bc20 msgr2=0x7f88840aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888a413700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888406bc20 0x7f88840aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888a413700 1 -- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870005200 msgr2=0x7f88840ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:51.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888a413700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870005200 0x7f88840ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888a413700 1 -- 172.21.15.92:0/2115928890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840af250 con 0x7f8884158ab0 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888a413700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8884158ab0 0x7f888406aec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8884070860 tx=0x7f887c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f887a7fc700 1 -- 172.21.15.92:0/2115928890 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c007000 con 0x7f8884158ab0 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f887a7fc700 1 -- 172.21.15.92:0/2115928890 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c003e30 con 0x7f8884158ab0 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888406ce40 con 0x7f8884158ab0 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f887a7fc700 1 -- 172.21.15.92:0/2115928890 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c014ab0 con 0x7f8884158ab0 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.902+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8884159100 con 0x7f8884158ab0 2023-06-09T04:35:51.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.903+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888404e8e0 con 0x7f8884158ab0 2023-06-09T04:35:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.903+0000 7f887a7fc700 1 -- 172.21.15.92:0/2115928890 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f887c023020 con 0x7f8884158ab0 2023-06-09T04:35:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.904+0000 7f887a7fc700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f886806ace0 0x7f886806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.904+0000 7f887a7fc700 1 -- 172.21.15.92:0/2115928890 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f887c08a810 con 0x7f8884158ab0 2023-06-09T04:35:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.904+0000 7f8889c12700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f886806ace0 0x7f886806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.904+0000 7f8889c12700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f886806ace0 0x7f886806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8874001710 tx=0x7f8874009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:51.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:51.908+0000 7f887a7fc700 1 -- 172.21.15.92:0/2115928890 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f887c055a70 con 0x7f8884158ab0 2023-06-09T04:35:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.096+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8884070890 con 0x7f8884158ab0 2023-06-09T04:35:52.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.097+0000 7f887a7fc700 1 -- 172.21.15.92:0/2115928890 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f887c059090 con 0x7f8884158ab0 2023-06-09T04:35:52.098 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:52.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f886806ace0 msgr2=0x7f886806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f886806ace0 0x7f886806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8874001710 tx=0x7f8874009040 comp rx=0 tx=0).stop 2023-06-09T04:35:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8884158ab0 msgr2=0x7f888406aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8884158ab0 0x7f888406aec0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8884070860 tx=0x7f887c004b40 comp rx=0 tx=0).stop 2023-06-09T04:35:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 shutdown_connections 2023-06-09T04:35:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f886806ace0 0x7f886806d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8884158ab0 0x7f888406aec0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8870005200 0x7f88840ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 --2- 172.21.15.92:0/2115928890 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f888406bc20 0x7f88840aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 >> 172.21.15.92:0/2115928890 conn(0x7f888406e4d0 msgr2=0x7f88840ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 shutdown_connections 2023-06-09T04:35:52.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.099+0000 7f888be76700 1 -- 172.21.15.92:0/2115928890 wait complete. 2023-06-09T04:35:52.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:52.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.235+0000 7f39a00af700 1 Processor -- start 2023-06-09T04:35:52.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f39a00af700 1 -- start start 2023-06-09T04:35:52.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f39a00af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3998069db0 0x7f399815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f39a00af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 0x7f399806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f39a00af700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399815d040 con 0x7f39980cf440 2023-06-09T04:35:52.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f39a00af700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3998160ab0 con 0x7f3998069db0 2023-06-09T04:35:52.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f39a00af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399815ca10 con 0x7f39980cce60 2023-06-09T04:35:52.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f399d64a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f39980cf440 0x7f399806a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37150/0 (socket says 172.21.15.92:37150) 2023-06-09T04:35:52.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.236+0000 7f399d64a700 1 -- 172.21.15.92:0/632366955 learned_addr learned my addr 172.21.15.92:0/632366955 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:52.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435756590 0 0) 0x7f399815d040 con 0x7f39980cf440 2023-06-09T04:35:52.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399e64c700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 0x7f399806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39840035b0 con 0x7f39980cf440 2023-06-09T04:35:52.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 --2- 172.21.15.92:0/632366955 >> v2:172.21.15.123:3300/0 conn(0x7f3998069db0 0x7f399815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945411195 0 0) 0x7f39840035b0 con 0x7f39980cf440 2023-06-09T04:35:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 msgr2=0x7f399806ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 0x7f399806ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 >> v2:172.21.15.123:3300/0 conn(0x7f3998069db0 msgr2=0x7f399815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 --2- 172.21.15.92:0/632366955 >> v2:172.21.15.123:3300/0 conn(0x7f3998069db0 0x7f399815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998160d50 con 0x7f39980cf440 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f3994001670 con 0x7f39980cf440 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 >> v1:172.21.15.92:6789/0 conn(0x7f39980cf440 legacy=0x7f399806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3984003a00 0x7f3984003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984004430 0x7f39840048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 0x7f3984005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3998160ab0 con 0x7f3984003a00 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399815ca10 con 0x7f3984004430 2023-06-09T04:35:52.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398400ba10 con 0x7f39980cce60 2023-06-09T04:35:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3984003a00 0x7f3984003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399d64a700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984004430 0x7f39840048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399e64c700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 0x7f3984005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 -- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 msgr2=0x7f3984005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39980cce60 0x7f3984005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 -- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984004430 msgr2=0x7f39840048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:52.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984004430 0x7f39840048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 -- 172.21.15.92:0/632366955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840035b0 con 0x7f3984003a00 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.237+0000 7f399de4b700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3984003a00 0x7f3984003e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b90 tx=0x7f3984005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbfdfda5f05e094 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398800e6c0 con 0x7f3984003a00 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398800e820 con 0x7f3984003a00 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f399ce49700 1 -- 172.21.15.92:0/632366955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398800eb90 con 0x7f3984003a00 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f39a00af700 1 -- 172.21.15.92:0/632366955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3984003a00 msgr2=0x7f3984003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f39a00af700 1 --2- 172.21.15.92:0/632366955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3984003a00 0x7f3984003e10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3988009b90 tx=0x7f3984005880 comp rx=0 tx=0).stop 2023-06-09T04:35:52.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f399e64c700 1 -- 172.21.15.92:0/632366955 reap_dead start 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f39a00af700 1 -- 172.21.15.92:0/632366955 shutdown_connections 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f39a00af700 1 -- 172.21.15.92:0/632366955 >> 172.21.15.92:0/632366955 conn(0x7f39980cc380 msgr2=0x7f39980cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f39a00af700 1 -- 172.21.15.92:0/632366955 shutdown_connections 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.238+0000 7f39a00af700 1 -- 172.21.15.92:0/632366955 wait complete. 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.239+0000 7f39a00af700 1 Processor -- start 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.239+0000 7f39a00af700 1 -- start start 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f39a00af700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3984003a00 0x7f39980b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f39a00af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3998154000 0x7f39980c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f39a00af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3998154470 0x7f39980c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f39a00af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980b4cd0 con 0x7f3998154000 2023-06-09T04:35:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f39a00af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39980b2230 con 0x7f3998154470 2023-06-09T04:35:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f39a00af700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39980b3950 con 0x7f3984003a00 2023-06-09T04:35:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f399d64a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3998154000 0x7f39980c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f399d64a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3998154000 0x7f39980c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38832/0 (socket says 172.21.15.92:38832) 2023-06-09T04:35:52.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f399d64a700 1 -- 172.21.15.92:0/2672833631 learned_addr learned my addr 172.21.15.92:0/2672833631 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f399de4b700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3984003a00 0x7f39980b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.240+0000 7f399e64c700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3998154470 0x7f39980c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f399d64a700 1 -- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3984003a00 msgr2=0x7f39980b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f399d64a700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3984003a00 0x7f39980b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f399d64a700 1 -- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3998154470 msgr2=0x7f39980c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f399d64a700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3998154470 0x7f39980c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f399d64a700 1 -- 172.21.15.92:0/2672833631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980b3c30 con 0x7f3998154000 2023-06-09T04:35:52.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f399e64c700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3998154470 0x7f39980c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f399d64a700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3998154000 0x7f39980c5840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3994001200 tx=0x7f399400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f398e7fc700 1 -- 172.21.15.92:0/2672833631 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994012590 con 0x7f3998154000 2023-06-09T04:35:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f398e7fc700 1 -- 172.21.15.92:0/2672833631 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994012710 con 0x7f3998154000 2023-06-09T04:35:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39980b3f10 con 0x7f3998154000 2023-06-09T04:35:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f398e7fc700 1 -- 172.21.15.92:0/2672833631 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994012ac0 con 0x7f3998154000 2023-06-09T04:35:52.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.241+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39980b44d0 con 0x7f3998154000 2023-06-09T04:35:52.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.242+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399806be50 con 0x7f3998154000 2023-06-09T04:35:52.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.242+0000 7f398e7fc700 1 -- 172.21.15.92:0/2672833631 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3994012c20 con 0x7f3998154000 2023-06-09T04:35:52.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.243+0000 7f398e7fc700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f397c06ace0 0x7f397c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.243+0000 7f399de4b700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f397c06ace0 0x7f397c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.243+0000 7f398e7fc700 1 -- 172.21.15.92:0/2672833631 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3994089250 con 0x7f3998154000 2023-06-09T04:35:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.243+0000 7f399de4b700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f397c06ace0 0x7f397c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39880090b0 tx=0x7f398801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.247+0000 7f398e7fc700 1 -- 172.21.15.92:0/2672833631 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3994054300 con 0x7f3998154000 2023-06-09T04:35:52.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.433+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f399804e8e0 con 0x7f3998154000 2023-06-09T04:35:52.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.434+0000 7f398e7fc700 1 -- 172.21.15.92:0/2672833631 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3994012ed0 con 0x7f3998154000 2023-06-09T04:35:52.435 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:52.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f397c06ace0 msgr2=0x7f397c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f397c06ace0 0x7f397c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39880090b0 tx=0x7f398801f040 comp rx=0 tx=0).stop 2023-06-09T04:35:52.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3998154000 msgr2=0x7f39980c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3998154000 0x7f39980c5840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3994001200 tx=0x7f399400ead0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 shutdown_connections 2023-06-09T04:35:52.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f397c06ace0 0x7f397c06d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3998154470 0x7f39980c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3984003a00 0x7f39980b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 --2- 172.21.15.92:0/2672833631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3998154000 0x7f39980c5840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 >> 172.21.15.92:0/2672833631 conn(0x7f39980cc380 msgr2=0x7f39980cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:52.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 shutdown_connections 2023-06-09T04:35:52.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.436+0000 7f39a00af700 1 -- 172.21.15.92:0/2672833631 wait complete. 2023-06-09T04:35:52.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:52.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.570+0000 7f236a132700 1 Processor -- start 2023-06-09T04:35:52.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.570+0000 7f236a132700 1 -- start start 2023-06-09T04:35:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.571+0000 7f236a132700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23641569f0 0x7f236415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.571+0000 7f236a132700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23640b2b00 0x7f23640ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.571+0000 7f236a132700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236415d270 con 0x7f23640b2f90 2023-06-09T04:35:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.571+0000 7f236a132700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236415d550 con 0x7f23640b2b00 2023-06-09T04:35:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.571+0000 7f236a132700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236415d830 con 0x7f23641569f0 2023-06-09T04:35:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.571+0000 7f2362ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f23640b2f90 0x7f23640b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37170/0 (socket says 172.21.15.92:37170) 2023-06-09T04:35:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.571+0000 7f2362ffd700 1 -- 172.21.15.92:0/3180746107 learned_addr learned my addr 172.21.15.92:0/3180746107 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23637fe700 1 --2- 172.21.15.92:0/3180746107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23641569f0 0x7f236415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f2363fff700 1 --2- 172.21.15.92:0/3180746107 >> v2:172.21.15.123:3300/0 conn(0x7f23640b2b00 0x7f23640ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23627fc700 1 -- 172.21.15.92:0/3180746107 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358426459 0 0) 0x7f236415d270 con 0x7f23640b2f90 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23627fc700 1 -- 172.21.15.92:0/3180746107 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f234c0035b0 con 0x7f23640b2f90 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23627fc700 1 -- 172.21.15.92:0/3180746107 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1557499448 0 0) 0x7f234c0035b0 con 0x7f23640b2f90 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23637fe700 1 -- 172.21.15.92:0/3180746107 >> v2:172.21.15.123:3300/0 conn(0x7f23640b2b00 msgr2=0x7f23640ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23637fe700 1 --2- 172.21.15.92:0/3180746107 >> v2:172.21.15.123:3300/0 conn(0x7f23640b2b00 0x7f23640ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23637fe700 1 -- 172.21.15.92:0/3180746107 >> v1:172.21.15.92:6789/0 conn(0x7f23640b2f90 legacy=0x7f23640b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23637fe700 1 -- 172.21.15.92:0/3180746107 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236406af60 con 0x7f23641569f0 2023-06-09T04:35:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23637fe700 1 --2- 172.21.15.92:0/3180746107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23641569f0 0x7f236415c9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2358009840 tx=0x7f2358009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8bbffdcfa6f3e7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23627fc700 1 -- 172.21.15.92:0/3180746107 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354001670 con 0x7f23641569f0 2023-06-09T04:35:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23627fc700 1 -- 172.21.15.92:0/3180746107 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2358004030 con 0x7f23641569f0 2023-06-09T04:35:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.572+0000 7f23627fc700 1 -- 172.21.15.92:0/3180746107 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580043c0 con 0x7f23641569f0 2023-06-09T04:35:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 -- 172.21.15.92:0/3180746107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23641569f0 msgr2=0x7f236415c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 --2- 172.21.15.92:0/3180746107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23641569f0 0x7f236415c9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2358009840 tx=0x7f2358009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 -- 172.21.15.92:0/3180746107 shutdown_connections 2023-06-09T04:35:52.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 --2- 172.21.15.92:0/3180746107 >> v2:172.21.15.123:3300/0 conn(0x7f23640b2b00 0x7f23640ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 --2- 172.21.15.92:0/3180746107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23641569f0 0x7f236415c9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 -- 172.21.15.92:0/3180746107 >> 172.21.15.92:0/3180746107 conn(0x7f236406e4d0 msgr2=0x7f236406e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 -- 172.21.15.92:0/3180746107 shutdown_connections 2023-06-09T04:35:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.573+0000 7f236a132700 1 -- 172.21.15.92:0/3180746107 wait complete. 2023-06-09T04:35:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.574+0000 7f236a132700 1 Processor -- start 2023-06-09T04:35:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.574+0000 7f236a132700 1 -- start start 2023-06-09T04:35:52.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f236a132700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23640b2b00 0x7f23641f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f236a132700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23640b2f90 0x7f23640aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f236a132700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23641569f0 0x7f23640c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f236a132700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23641f6460 con 0x7f23640b2b00 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f236a132700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23640c63e0 con 0x7f23641569f0 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f236a132700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23640af500 con 0x7f23640b2f90 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23640b2b00 0x7f23641f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23640b2b00 0x7f23641f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38842/0 (socket says 172.21.15.92:38842) 2023-06-09T04:35:52.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 -- 172.21.15.92:0/3722698701 learned_addr learned my addr 172.21.15.92:0/3722698701 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f2362ffd700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23640b2f90 0x7f23640aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f2363fff700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23641569f0 0x7f23640c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 -- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23640b2f90 msgr2=0x7f23640aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23640b2f90 0x7f23640aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 -- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23641569f0 msgr2=0x7f23640c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23641569f0 0x7f23640c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 -- 172.21.15.92:0/3722698701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580094a0 con 0x7f23640b2b00 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f2363fff700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23641569f0 0x7f23640c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.575+0000 7f23637fe700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23640b2b00 0x7f23641f5f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f235800a010 tx=0x7f2358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.576+0000 7f234bfff700 1 -- 172.21.15.92:0/3722698701 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358003c40 con 0x7f23640b2b00 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.576+0000 7f234bfff700 1 -- 172.21.15.92:0/3722698701 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235801f070 con 0x7f23640b2b00 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.576+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23640c54f0 con 0x7f23640b2b00 2023-06-09T04:35:52.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.576+0000 7f234bfff700 1 -- 172.21.15.92:0/3722698701 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358011630 con 0x7f23640b2b00 2023-06-09T04:35:52.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.576+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23640c5ab0 con 0x7f23640b2b00 2023-06-09T04:35:52.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.577+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23640c66c0 con 0x7f23640b2b00 2023-06-09T04:35:52.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.577+0000 7f234bfff700 1 -- 172.21.15.92:0/3722698701 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2358021030 con 0x7f23640b2b00 2023-06-09T04:35:52.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.577+0000 7f234bfff700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f235006ace0 0x7f235006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.578+0000 7f234bfff700 1 -- 172.21.15.92:0/3722698701 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f235808a830 con 0x7f23640b2b00 2023-06-09T04:35:52.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.578+0000 7f2362ffd700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f235006ace0 0x7f235006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.578+0000 7f2362ffd700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f235006ace0 0x7f235006d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2354001eb0 tx=0x7f2354001ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.582+0000 7f234bfff700 1 -- 172.21.15.92:0/3722698701 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2358055970 con 0x7f23640b2b00 2023-06-09T04:35:52.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.766+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f236404e8e0 con 0x7f23640b2b00 2023-06-09T04:35:52.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.767+0000 7f234bfff700 1 -- 172.21.15.92:0/3722698701 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2358058f90 con 0x7f23640b2b00 2023-06-09T04:35:52.768 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:52.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f235006ace0 msgr2=0x7f235006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f235006ace0 0x7f235006d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2354001eb0 tx=0x7f2354001ce0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23640b2b00 msgr2=0x7f23641f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23640b2b00 0x7f23641f5f20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f235800a010 tx=0x7f2358004750 comp rx=0 tx=0).stop 2023-06-09T04:35:52.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 shutdown_connections 2023-06-09T04:35:52.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f235006ace0 0x7f235006d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23641569f0 0x7f23640c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23640b2b00 0x7f23641f5f20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 --2- 172.21.15.92:0/3722698701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f23640b2f90 0x7f23640aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 >> 172.21.15.92:0/3722698701 conn(0x7f236406e4d0 msgr2=0x7f236406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:52.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 shutdown_connections 2023-06-09T04:35:52.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.769+0000 7f236a132700 1 -- 172.21.15.92:0/3722698701 wait complete. 2023-06-09T04:35:52.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-09T04:35:52.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.902+0000 7f6329a4d700 1 Processor -- start 2023-06-09T04:35:52.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.902+0000 7f6329a4d700 1 -- start start 2023-06-09T04:35:52.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f6329a4d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f632415c850 0x7f632406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f6329a4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f63240cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f6329a4d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632406ef00 con 0x7f63240cfe60 2023-06-09T04:35:52.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f6329a4d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632406f1a0 con 0x7f632415c850 2023-06-09T04:35:52.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f6329a4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632406f480 con 0x7f632406b7e0 2023-06-09T04:35:52.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f63237fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f63240cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f6322ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f632415c850 0x7f632406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f63237fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f63240cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:47764/0 (socket says 172.21.15.92:47764) 2023-06-09T04:35:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f63227fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f63240cfe60 0x7f632406b0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59680/0 (socket says 172.21.15.92:59680) 2023-06-09T04:35:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.903+0000 7f63237fe700 1 -- 172.21.15.92:0/1560898559 learned_addr learned my addr 172.21.15.92:0/1560898559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6322ffd700 1 -- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 msgr2=0x7f63240cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6322ffd700 1 --2- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f63240cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6322ffd700 1 -- 172.21.15.92:0/1560898559 >> v1:172.21.15.123:6789/0 conn(0x7f63240cfe60 legacy=0x7f632406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6321ffb700 1 -- 172.21.15.92:0/1560898559 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379325478 0 0) 0x7f632406ef00 con 0x7f63240cfe60 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6322ffd700 1 -- 172.21.15.92:0/1560898559 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632406f760 con 0x7f632415c850 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f63237fe700 1 --2- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f63240cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6322ffd700 1 --2- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 0x7f632406ff60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6318009840 tx=0x7f6318009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7e6f0e1962a0ba server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6321ffb700 1 -- 172.21.15.92:0/1560898559 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318011830 con 0x7f632415c850 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6321ffb700 1 -- 172.21.15.92:0/1560898559 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6318011990 con 0x7f632415c850 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.904+0000 7f6321ffb700 1 -- 172.21.15.92:0/1560898559 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318011d20 con 0x7f632415c850 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 -- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 msgr2=0x7f632406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 --2- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 0x7f632406ff60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6318009840 tx=0x7f6318009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:52.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 -- 172.21.15.92:0/1560898559 shutdown_connections 2023-06-09T04:35:52.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 --2- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f63240cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 --2- 172.21.15.92:0/1560898559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 0x7f632406ff60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 -- 172.21.15.92:0/1560898559 >> 172.21.15.92:0/1560898559 conn(0x7f6324089fe0 msgr2=0x7f63240d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:52.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 -- 172.21.15.92:0/1560898559 shutdown_connections 2023-06-09T04:35:52.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.905+0000 7f6329a4d700 1 -- 172.21.15.92:0/1560898559 wait complete. 2023-06-09T04:35:52.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.906+0000 7f6329a4d700 1 Processor -- start 2023-06-09T04:35:52.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.907+0000 7f6329a4d700 1 -- start start 2023-06-09T04:35:52.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.907+0000 7f6329a4d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f6324159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.907+0000 7f6329a4d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63240cfe60 0x7f632415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.907+0000 7f6329a4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 0x7f63240c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.907+0000 7f6329a4d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632415a1b0 con 0x7f63240cfe60 2023-06-09T04:35:52.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.907+0000 7f6329a4d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632415bb90 con 0x7f632415c850 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.907+0000 7f6329a4d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632415b5d0 con 0x7f632406b7e0 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63240cfe60 0x7f632415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63240cfe60 0x7f632415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38860/0 (socket says 172.21.15.92:38860) 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 -- 172.21.15.92:0/2310150043 learned_addr learned my addr 172.21.15.92:0/2310150043 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f6322ffd700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f6324159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63237fe700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 0x7f63240c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 -- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 msgr2=0x7f6324159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f6324159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 -- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 msgr2=0x7f63240c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 0x7f63240c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 -- 172.21.15.92:0/2310150043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c009b30 con 0x7f63240cfe60 2023-06-09T04:35:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f63227fc700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63240cfe60 0x7f632415ad80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63140011e0 tx=0x7f631400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f630bfff700 1 -- 172.21.15.92:0/2310150043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314004c80 con 0x7f63240cfe60 2023-06-09T04:35:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f630bfff700 1 -- 172.21.15.92:0/2310150043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6314004e00 con 0x7f63240cfe60 2023-06-09T04:35:52.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63180094a0 con 0x7f63240cfe60 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f630bfff700 1 -- 172.21.15.92:0/2310150043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314007310 con 0x7f63240cfe60 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.908+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63240cc010 con 0x7f63240cfe60 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.909+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63240740e0 con 0x7f63240cfe60 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.910+0000 7f630bfff700 1 -- 172.21.15.92:0/2310150043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6314007470 con 0x7f63240cfe60 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.910+0000 7f630bfff700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f631006ace0 0x7f631006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.910+0000 7f630bfff700 1 -- 172.21.15.92:0/2310150043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6314089140 con 0x7f63240cfe60 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.910+0000 7f6322ffd700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f631006ace0 0x7f631006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:52.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.910+0000 7f6322ffd700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f631006ace0 0x7f631006d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63180090b0 tx=0x7f631801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:52.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:52.914+0000 7f630bfff700 1 -- 172.21.15.92:0/2310150043 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6314054170 con 0x7f63240cfe60 2023-06-09T04:35:53.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.113+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f632415e5e0 con 0x7f63240cfe60 2023-06-09T04:35:53.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.114+0000 7f630bfff700 1 -- 172.21.15.92:0/2310150043 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6314057790 con 0x7f63240cfe60 2023-06-09T04:35:53.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f631006ace0 msgr2=0x7f631006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f631006ace0 0x7f631006d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63180090b0 tx=0x7f631801c040 comp rx=0 tx=0).stop 2023-06-09T04:35:53.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63240cfe60 msgr2=0x7f632415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63240cfe60 0x7f632415ad80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f63140011e0 tx=0x7f631400ca30 comp rx=0 tx=0).stop 2023-06-09T04:35:53.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 shutdown_connections 2023-06-09T04:35:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f631006ace0 0x7f631006d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f632415c850 0x7f63240c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f632406b7e0 0x7f6324159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 --2- 172.21.15.92:0/2310150043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63240cfe60 0x7f632415ad80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 >> 172.21.15.92:0/2310150043 conn(0x7f6324089fe0 msgr2=0x7f632415d250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 shutdown_connections 2023-06-09T04:35:53.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.116+0000 7f6329a4d700 1 -- 172.21.15.92:0/2310150043 wait complete. 2023-06-09T04:35:53.119 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-06-09T04:35:53.129 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-06-09T04:35:53.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:53.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-06-09T04:35:53.180 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:35:53.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:53.180 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-09T04:35:53.202 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:53.203 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:53.219 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:53.220 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:53.277 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:35:53.278 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:53.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:53.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.491+0000 7f6b2e1d6700 1 Processor -- start 2023-06-09T04:35:53.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.491+0000 7f6b2e1d6700 1 -- start start 2023-06-09T04:35:53.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b2e1d6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6b2815c7d0 0x7f6b280ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b2e1d6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b280ce730 con 0x7f6b280cf010 2023-06-09T04:35:53.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b2e1d6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2806aaf0 con 0x7f6b280ac400 2023-06-09T04:35:53.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b2e1d6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2815cba0 con 0x7f6b2815c7d0 2023-06-09T04:35:53.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b27fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6b280cf010 0x7f6b2806e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37200/0 (socket says 172.21.15.92:37200) 2023-06-09T04:35:53.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b27fff700 1 -- 172.21.15.92:0/1379240272 learned_addr learned my addr 172.21.15.92:0/1379240272 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:53.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754479595 0 0) 0x7f6b20001490 con 0x7f6b280cf010 2023-06-09T04:35:53.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b277fe700 1 --2- 172.21.15.92:0/1379240272 >> v2:172.21.15.123:3300/0 conn(0x7f6b2815c7d0 0x7f6b280ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b140035b0 con 0x7f6b280cf010 2023-06-09T04:35:53.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1419513528 0 0) 0x7f6b140035b0 con 0x7f6b280cf010 2023-06-09T04:35:53.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 >> v2:172.21.15.123:3300/0 conn(0x7f6b2815c7d0 msgr2=0x7f6b280ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:53.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 --2- 172.21.15.92:0/1379240272 >> v2:172.21.15.123:3300/0 conn(0x7f6b2815c7d0 0x7f6b280ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 >> v1:172.21.15.123:6789/0 conn(0x7f6b280ac400 legacy=0x7f6b280ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28069ee0 con 0x7f6b280cf010 2023-06-09T04:35:53.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f6b20001b80 con 0x7f6b280cf010 2023-06-09T04:35:53.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b277fe700 1 --2- 172.21.15.92:0/1379240272 >> v2:172.21.15.123:3300/0 conn(0x7f6b2815c7d0 0x7f6b280ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 >> v1:172.21.15.92:6789/0 conn(0x7f6b280cf010 legacy=0x7f6b2806e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b14003a00 0x7f6b14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.492+0000 7f6b267fc700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b14004850 0x7f6b14004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b267fc700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b1400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2815cba0 con 0x7f6b14005320 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1400c1e0 con 0x7f6b14003a00 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1400c3a0 con 0x7f6b14004850 2023-06-09T04:35:53.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b1400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b277fe700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b14003a00 0x7f6b14003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b26ffd700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b14004850 0x7f6b14004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 -- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b14004850 msgr2=0x7f6b14004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b14004850 0x7f6b14004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 -- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b14003a00 msgr2=0x7f6b14003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b14003a00 0x7f6b14003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 -- 172.21.15.92:0/1379240272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b140035b0 con 0x7f6b14005320 2023-06-09T04:35:53.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b26ffd700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b14004850 0x7f6b14004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:53.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 -- 172.21.15.92:0/1379240272 reap_dead start 2023-06-09T04:35:53.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b27fff700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b1400ba10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b2808a0d0 tx=0x7f6b14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62f5ed7473c26cd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:53.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000a470 con 0x7f6b14005320 2023-06-09T04:35:53.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b20012620 con 0x7f6b14005320 2023-06-09T04:35:53.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.493+0000 7f6b267fc700 1 -- 172.21.15.92:0/1379240272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200129d0 con 0x7f6b14005320 2023-06-09T04:35:53.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.494+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/1379240272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 msgr2=0x7f6b1400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.494+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b1400ba10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b2808a0d0 tx=0x7f6b14005880 comp rx=0 tx=0).stop 2023-06-09T04:35:53.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.494+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/1379240272 shutdown_connections 2023-06-09T04:35:53.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.494+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/1379240272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b1400ba10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.494+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/1379240272 >> 172.21.15.92:0/1379240272 conn(0x7f6b28098c10 msgr2=0x7f6b280cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:53.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.494+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/1379240272 shutdown_connections 2023-06-09T04:35:53.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.494+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/1379240272 wait complete. 2023-06-09T04:35:53.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.495+0000 7f6b2e1d6700 1 Processor -- start 2023-06-09T04:35:53.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.495+0000 7f6b2e1d6700 1 -- start start 2023-06-09T04:35:53.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b2e1d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b280cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b2e1d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b280c5550 0x7f6b280c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b2e1d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b28099580 0x7f6b280999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b2e1d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b28148230 con 0x7f6b14005320 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b2e1d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b280c5f00 con 0x7f6b280c5550 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b2e1d6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2809ad90 con 0x7f6b28099580 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b280cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.496+0000 7f6b277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b280cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38876/0 (socket says 172.21.15.92:38876) 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b277fe700 1 -- 172.21.15.92:0/2761784409 learned_addr learned my addr 172.21.15.92:0/2761784409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:53.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b27fff700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b28099580 0x7f6b280999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b26ffd700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b280c5550 0x7f6b280c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b277fe700 1 -- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b28099580 msgr2=0x7f6b280999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b277fe700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b28099580 0x7f6b280999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b277fe700 1 -- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b280c5550 msgr2=0x7f6b280c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b277fe700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b280c5550 0x7f6b280c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b277fe700 1 -- 172.21.15.92:0/2761784409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280cd1f0 con 0x7f6b14005320 2023-06-09T04:35:53.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b26ffd700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b280c5550 0x7f6b280c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b277fe700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b280cccb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6b100116d0 tx=0x7f6b100119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b1ffff700 1 -- 172.21.15.92:0/2761784409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000c070 con 0x7f6b14005320 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b1ffff700 1 -- 172.21.15.92:0/2761784409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b10015870 con 0x7f6b14005320 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2809a2b0 con 0x7f6b14005320 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b1ffff700 1 -- 172.21.15.92:0/2761784409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10015c00 con 0x7f6b14005320 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.497+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b28147e50 con 0x7f6b14005320 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.498+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b281484a0 con 0x7f6b14005320 2023-06-09T04:35:53.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.499+0000 7f6b1ffff700 1 -- 172.21.15.92:0/2761784409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b10013400 con 0x7f6b14005320 2023-06-09T04:35:53.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.499+0000 7f6b1ffff700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6b0806ace0 0x7f6b0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.499+0000 7f6b1ffff700 1 -- 172.21.15.92:0/2761784409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b10089620 con 0x7f6b14005320 2023-06-09T04:35:53.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.499+0000 7f6b26ffd700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6b0806ace0 0x7f6b0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.499+0000 7f6b26ffd700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6b0806ace0 0x7f6b0806d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b18000c00 tx=0x7f6b18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:53.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.503+0000 7f6b1ffff700 1 -- 172.21.15.92:0/2761784409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6b100546d0 con 0x7f6b14005320 2023-06-09T04:35:53.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.692+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b281479d0 con 0x7f6b14005320 2023-06-09T04:35:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.693+0000 7f6b1ffff700 1 -- 172.21.15.92:0/2761784409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f6b281479d0 con 0x7f6b14005320 2023-06-09T04:35:53.694 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:53.695 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:53.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6b0806ace0 msgr2=0x7f6b0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6b0806ace0 0x7f6b0806d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b18000c00 tx=0x7f6b18009040 comp rx=0 tx=0).stop 2023-06-09T04:35:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 msgr2=0x7f6b280cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b280cccb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6b100116d0 tx=0x7f6b100119e0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 shutdown_connections 2023-06-09T04:35:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6b0806ace0 0x7f6b0806d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6b28099580 0x7f6b280999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b14005320 0x7f6b280cccb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 --2- 172.21.15.92:0/2761784409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b280c5550 0x7f6b280c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 >> 172.21.15.92:0/2761784409 conn(0x7f6b28098c10 msgr2=0x7f6b280b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 shutdown_connections 2023-06-09T04:35:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.695+0000 7f6b2e1d6700 1 -- 172.21.15.92:0/2761784409 wait complete. 2023-06-09T04:35:53.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:53.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.830+0000 7f82cb7f0700 1 Processor -- start 2023-06-09T04:35:53.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.830+0000 7f82cb7f0700 1 -- start start 2023-06-09T04:35:53.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82cb7f0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82c4069db0 0x7f82c415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82cb7f0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cf440 0x7f82c406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82cb7f0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c415d040 con 0x7f82c40cce60 2023-06-09T04:35:53.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82cb7f0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c4160ab0 con 0x7f82c4069db0 2023-06-09T04:35:53.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82cb7f0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415ca10 con 0x7f82c40cf440 2023-06-09T04:35:53.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c958c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f82c4069db0 0x7f82c415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c9d8d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f82c40cce60 0x7f82c406ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37220/0 (socket says 172.21.15.92:37220) 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c9d8d700 1 -- 172.21.15.92:0/4079978880 learned_addr learned my addr 172.21.15.92:0/4079978880 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c8d8b700 1 --2- 172.21.15.92:0/4079978880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cf440 0x7f82c406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c958c700 1 -- 172.21.15.92:0/4079978880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cf440 msgr2=0x7f82c406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c958c700 1 --2- 172.21.15.92:0/4079978880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cf440 0x7f82c406a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c958c700 1 -- 172.21.15.92:0/4079978880 >> v1:172.21.15.92:6789/0 conn(0x7f82c40cce60 legacy=0x7f82c406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82c958c700 1 -- 172.21.15.92:0/4079978880 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c406a6c0 con 0x7f82c4069db0 2023-06-09T04:35:53.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.831+0000 7f82bbfff700 1 -- 172.21.15.92:0/4079978880 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796529640 0 0) 0x7f82c415d040 con 0x7f82c40cce60 2023-06-09T04:35:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82c958c700 1 --2- 172.21.15.92:0/4079978880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c4069db0 0x7f82c415d4d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009840 tx=0x7f82b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71af9e05baa27c98 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82bbfff700 1 -- 172.21.15.92:0/4079978880 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011830 con 0x7f82c4069db0 2023-06-09T04:35:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82bbfff700 1 -- 172.21.15.92:0/4079978880 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4011990 con 0x7f82c4069db0 2023-06-09T04:35:53.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82bbfff700 1 -- 172.21.15.92:0/4079978880 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4011d20 con 0x7f82c4069db0 2023-06-09T04:35:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 -- 172.21.15.92:0/4079978880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c4069db0 msgr2=0x7f82c415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/4079978880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c4069db0 0x7f82c415d4d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f82b4009840 tx=0x7f82b4009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 -- 172.21.15.92:0/4079978880 shutdown_connections 2023-06-09T04:35:53.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/4079978880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c4069db0 0x7f82c415d4d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/4079978880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cf440 0x7f82c406a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 -- 172.21.15.92:0/4079978880 >> 172.21.15.92:0/4079978880 conn(0x7f82c40cc380 msgr2=0x7f82c40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 -- 172.21.15.92:0/4079978880 shutdown_connections 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.832+0000 7f82cb7f0700 1 -- 172.21.15.92:0/4079978880 wait complete. 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.833+0000 7f82cb7f0700 1 Processor -- start 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82cb7f0700 1 -- start start 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82cb7f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82c4069db0 0x7f82c40c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82cb7f0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cce60 0x7f82c40af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82cb7f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c40cf440 0x7f82c41e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82cb7f0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40b56c0 con 0x7f82c40cf440 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82cb7f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40b5100 con 0x7f82c4069db0 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82cb7f0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40b53e0 con 0x7f82c40cce60 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82c9d8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c40cf440 0x7f82c41e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82c9d8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c40cf440 0x7f82c41e7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38892/0 (socket says 172.21.15.92:38892) 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82c9d8d700 1 -- 172.21.15.92:0/223482068 learned_addr learned my addr 172.21.15.92:0/223482068 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:53.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.834+0000 7f82c958c700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82c4069db0 0x7f82c40c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c8d8b700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cce60 0x7f82c40af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c9d8d700 1 -- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cce60 msgr2=0x7f82c40af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c9d8d700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cce60 0x7f82c40af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c9d8d700 1 -- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82c4069db0 msgr2=0x7f82c40c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:53.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c9d8d700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82c4069db0 0x7f82c40c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:53.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c9d8d700 1 -- 172.21.15.92:0/223482068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b40094a0 con 0x7f82c40cf440 2023-06-09T04:35:53.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c8d8b700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cce60 0x7f82c40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c958c700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82c4069db0 0x7f82c40c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82c9d8d700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c40cf440 0x7f82c41e7f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00cbe0 tx=0x7f82bc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82b9ffb700 1 -- 172.21.15.92:0/223482068 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00abc0 con 0x7f82c40cf440 2023-06-09T04:35:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82b9ffb700 1 -- 172.21.15.92:0/223482068 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc00ad40 con 0x7f82c40cf440 2023-06-09T04:35:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82b9ffb700 1 -- 172.21.15.92:0/223482068 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc007310 con 0x7f82c40cf440 2023-06-09T04:35:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c40c6b70 con 0x7f82c40cf440 2023-06-09T04:35:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.835+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c40c8ab0 con 0x7f82c40cf440 2023-06-09T04:35:53.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.836+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c40c7090 con 0x7f82c40cf440 2023-06-09T04:35:53.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.837+0000 7f82b9ffb700 1 -- 172.21.15.92:0/223482068 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82bc007470 con 0x7f82c40cf440 2023-06-09T04:35:53.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.837+0000 7f82b9ffb700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f82a806ad10 0x7f82a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:53.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.837+0000 7f82b9ffb700 1 -- 172.21.15.92:0/223482068 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82bc088fb0 con 0x7f82c40cf440 2023-06-09T04:35:53.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.837+0000 7f82c958c700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f82a806ad10 0x7f82a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:53.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.837+0000 7f82c958c700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f82a806ad10 0x7f82a806d1c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82b40090b0 tx=0x7f82b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:53.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:53.841+0000 7f82b9ffb700 1 -- 172.21.15.92:0/223482068 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f82bc0bb310 con 0x7f82c40cf440 2023-06-09T04:35:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.027+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82c40c69b0 con 0x7f82c40cf440 2023-06-09T04:35:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.027+0000 7f82b9ffb700 1 -- 172.21.15.92:0/223482068 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82bc0180a0 con 0x7f82c40cf440 2023-06-09T04:35:54.029 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f82a806ad10 msgr2=0x7f82a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f82a806ad10 0x7f82a806d1c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82b40090b0 tx=0x7f82b4008030 comp rx=0 tx=0).stop 2023-06-09T04:35:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c40cf440 msgr2=0x7f82c41e7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c40cf440 0x7f82c41e7f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00cbe0 tx=0x7f82bc00a060 comp rx=0 tx=0).stop 2023-06-09T04:35:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 shutdown_connections 2023-06-09T04:35:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f82a806ad10 0x7f82a806d1c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f82c40cf440 0x7f82c41e7f10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82c4069db0 0x7f82c40c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 --2- 172.21.15.92:0/223482068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82c40cce60 0x7f82c40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 >> 172.21.15.92:0/223482068 conn(0x7f82c40cc380 msgr2=0x7f82c4071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 shutdown_connections 2023-06-09T04:35:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.029+0000 7f82cb7f0700 1 -- 172.21.15.92:0/223482068 wait complete. 2023-06-09T04:35:54.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:54.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.163+0000 7fc674b48700 1 Processor -- start 2023-06-09T04:35:54.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.163+0000 7fc674b48700 1 -- start start 2023-06-09T04:35:54.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc674b48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6700ce3d0 0x7fc6700713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc674b48700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc6700aed50 0x7fc6700ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc674b48700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc67015d470 0x7fc67015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc674b48700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc670072350 con 0x7fc67015d470 2023-06-09T04:35:54.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc674b48700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6700ce190 con 0x7fc6700aed50 2023-06-09T04:35:54.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc674b48700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700cece0 con 0x7fc6700ce3d0 2023-06-09T04:35:54.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc66ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc67015d470 0x7fc67015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc66ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc67015d470 0x7fc67015d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38900/0 (socket says 172.21.15.92:38900) 2023-06-09T04:35:54.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.164+0000 7fc66ed9d700 1 -- 172.21.15.92:0/3622710448 learned_addr learned my addr 172.21.15.92:0/3622710448 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:54.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66ed9d700 1 -- 172.21.15.92:0/3622710448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6700ce3d0 msgr2=0x7fc6700713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:54.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66e59c700 1 --2- 172.21.15.92:0/3622710448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6700ce3d0 0x7fc6700713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66dd9b700 1 --2- 172.21.15.92:0/3622710448 >> v2:172.21.15.123:3300/0 conn(0x7fc6700aed50 0x7fc6700ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66ed9d700 1 --2- 172.21.15.92:0/3622710448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6700ce3d0 0x7fc6700713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66ed9d700 1 -- 172.21.15.92:0/3622710448 >> v2:172.21.15.123:3300/0 conn(0x7fc6700aed50 msgr2=0x7fc6700ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66ed9d700 1 --2- 172.21.15.92:0/3622710448 >> v2:172.21.15.123:3300/0 conn(0x7fc6700aed50 0x7fc6700ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66ed9d700 1 -- 172.21.15.92:0/3622710448 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc670071ab0 con 0x7fc67015d470 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66e59c700 1 --2- 172.21.15.92:0/3622710448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6700ce3d0 0x7fc6700713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66dd9b700 1 --2- 172.21.15.92:0/3622710448 >> v2:172.21.15.123:3300/0 conn(0x7fc6700aed50 0x7fc6700ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66ed9d700 1 --2- 172.21.15.92:0/3622710448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc67015d470 0x7fc67015d8f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6700710a0 tx=0x7fc66800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af375478f63b16e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66d59a700 1 -- 172.21.15.92:0/3622710448 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800e070 con 0x7fc67015d470 2023-06-09T04:35:54.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66d59a700 1 -- 172.21.15.92:0/3622710448 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66800b9c0 con 0x7fc67015d470 2023-06-09T04:35:54.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.165+0000 7fc66d59a700 1 -- 172.21.15.92:0/3622710448 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800bd70 con 0x7fc67015d470 2023-06-09T04:35:54.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 -- 172.21.15.92:0/3622710448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc67015d470 msgr2=0x7fc67015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 --2- 172.21.15.92:0/3622710448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc67015d470 0x7fc67015d8f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6700710a0 tx=0x7fc66800b330 comp rx=0 tx=0).stop 2023-06-09T04:35:54.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 -- 172.21.15.92:0/3622710448 shutdown_connections 2023-06-09T04:35:54.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 --2- 172.21.15.92:0/3622710448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc67015d470 0x7fc67015d8f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 --2- 172.21.15.92:0/3622710448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6700ce3d0 0x7fc6700713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 --2- 172.21.15.92:0/3622710448 >> v2:172.21.15.123:3300/0 conn(0x7fc6700aed50 0x7fc6700ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 -- 172.21.15.92:0/3622710448 >> 172.21.15.92:0/3622710448 conn(0x7fc6700cc380 msgr2=0x7fc6700cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:54.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 -- 172.21.15.92:0/3622710448 shutdown_connections 2023-06-09T04:35:54.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.166+0000 7fc674b48700 1 -- 172.21.15.92:0/3622710448 wait complete. 2023-06-09T04:35:54.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.167+0000 7fc674b48700 1 Processor -- start 2023-06-09T04:35:54.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.167+0000 7fc674b48700 1 -- start start 2023-06-09T04:35:54.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc674b48700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6700aed50 0x7fc6701f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc674b48700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6700ce3d0 0x7fc6701f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc674b48700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc67015d470 0x7fc6700c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc674b48700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700c6410 con 0x7fc6700aed50 2023-06-09T04:35:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc674b48700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700c5e50 con 0x7fc6700ce3d0 2023-06-09T04:35:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc674b48700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700c6130 con 0x7fc67015d470 2023-06-09T04:35:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6700aed50 0x7fc6701f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6700aed50 0x7fc6701f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38902/0 (socket says 172.21.15.92:38902) 2023-06-09T04:35:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 -- 172.21.15.92:0/135343056 learned_addr learned my addr 172.21.15.92:0/135343056 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66ed9d700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc67015d470 0x7fc6700c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66dd9b700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6700ce3d0 0x7fc6701f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 -- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc67015d470 msgr2=0x7fc6700c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc67015d470 0x7fc6700c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 -- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6700ce3d0 msgr2=0x7fc6701f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6700ce3d0 0x7fc6701f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 -- 172.21.15.92:0/135343056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668009b30 con 0x7fc6700aed50 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc66e59c700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6700aed50 0x7fc6701f62a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc66400c750 tx=0x7fc66400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.168+0000 7fc65effd700 1 -- 172.21.15.92:0/135343056 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664011800 con 0x7fc6700aed50 2023-06-09T04:35:54.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.169+0000 7fc65effd700 1 -- 172.21.15.92:0/135343056 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc664011960 con 0x7fc6700aed50 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.169+0000 7fc65effd700 1 -- 172.21.15.92:0/135343056 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664011cd0 con 0x7fc6700aed50 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.169+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6701f5aa0 con 0x7fc6700aed50 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.169+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6701f67e0 con 0x7fc6700aed50 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.169+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6701f5fc0 con 0x7fc6700aed50 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.170+0000 7fc65effd700 1 -- 172.21.15.92:0/135343056 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc664021350 con 0x7fc6700aed50 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.170+0000 7fc65effd700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc65006ace0 0x7fc65006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.171+0000 7fc66dd9b700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc65006ace0 0x7fc65006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.171+0000 7fc65effd700 1 -- 172.21.15.92:0/135343056 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc664089220 con 0x7fc6700aed50 2023-06-09T04:35:54.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.171+0000 7fc66dd9b700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc65006ace0 0x7fc65006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc67015c5e0 tx=0x7fc660008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.175+0000 7fc65effd700 1 -- 172.21.15.92:0/135343056 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc664057df0 con 0x7fc6700aed50 2023-06-09T04:35:54.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.362+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc67006be50 con 0x7fc6700aed50 2023-06-09T04:35:54.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.362+0000 7fc65effd700 1 -- 172.21.15.92:0/135343056 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc664057980 con 0x7fc6700aed50 2023-06-09T04:35:54.364 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc65006ace0 msgr2=0x7fc65006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc65006ace0 0x7fc65006d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc67015c5e0 tx=0x7fc660008040 comp rx=0 tx=0).stop 2023-06-09T04:35:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6700aed50 msgr2=0x7fc6701f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6700aed50 0x7fc6701f62a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc66400c750 tx=0x7fc66400ca60 comp rx=0 tx=0).stop 2023-06-09T04:35:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 shutdown_connections 2023-06-09T04:35:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc65006ace0 0x7fc65006d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc67015d470 0x7fc6700c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6700aed50 0x7fc6701f62a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 --2- 172.21.15.92:0/135343056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6700ce3d0 0x7fc6701f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 >> 172.21.15.92:0/135343056 conn(0x7fc6700cc380 msgr2=0x7fc670158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:54.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 shutdown_connections 2023-06-09T04:35:54.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.364+0000 7fc674b48700 1 -- 172.21.15.92:0/135343056 wait complete. 2023-06-09T04:35:54.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:54.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.499+0000 7f6c4af27700 1 Processor -- start 2023-06-09T04:35:54.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.499+0000 7f6c4af27700 1 -- start start 2023-06-09T04:35:54.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c4af27700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c440cda90 0x7f6c440ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c4af27700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c440cb700 con 0x7f6c440b60d0 2023-06-09T04:35:54.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c4af27700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c440b64a0 con 0x7f6c440d03b0 2023-06-09T04:35:54.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c4af27700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440ccf50 con 0x7f6c440cda90 2023-06-09T04:35:54.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c494c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c440cda90 0x7f6c440ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c48cc3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c440b60d0 0x7f6c4415e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37242/0 (socket says 172.21.15.92:37242) 2023-06-09T04:35:54.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c494c4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c440cda90 0x7f6c440ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38914/0 (socket says 172.21.15.92:38914) 2023-06-09T04:35:54.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c48cc3700 1 -- 172.21.15.92:0/1915319343 learned_addr learned my addr 172.21.15.92:0/1915319343 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:54.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c494c4700 1 -- 172.21.15.92:0/1915319343 >> v1:172.21.15.123:6789/0 conn(0x7f6c440d03b0 legacy=0x7f6c440cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:54.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c494c4700 1 -- 172.21.15.92:0/1915319343 >> v1:172.21.15.92:6789/0 conn(0x7f6c440b60d0 legacy=0x7f6c4415e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c494c4700 1 -- 172.21.15.92:0/1915319343 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440cb140 con 0x7f6c440cda90 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c437fe700 1 -- 172.21.15.92:0/1915319343 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117628347 0 0) 0x7f6c38001110 con 0x7f6c440b60d0 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c494c4700 1 --2- 172.21.15.92:0/1915319343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440cda90 0x7f6c440ce540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009d90 tx=0x7f6c3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e00e581a1b64b32 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c437fe700 1 -- 172.21.15.92:0/1915319343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00e070 con 0x7f6c440cda90 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c437fe700 1 -- 172.21.15.92:0/1915319343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c00bbd0 con 0x7f6c440cda90 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.500+0000 7f6c437fe700 1 -- 172.21.15.92:0/1915319343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c003a20 con 0x7f6c440cda90 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.501+0000 7f6c4af27700 1 -- 172.21.15.92:0/1915319343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440cda90 msgr2=0x7f6c440ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.501+0000 7f6c4af27700 1 --2- 172.21.15.92:0/1915319343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440cda90 0x7f6c440ce540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009d90 tx=0x7f6c3c00b330 comp rx=0 tx=0).stop 2023-06-09T04:35:54.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.501+0000 7f6c4af27700 1 -- 172.21.15.92:0/1915319343 shutdown_connections 2023-06-09T04:35:54.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.501+0000 7f6c4af27700 1 --2- 172.21.15.92:0/1915319343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440cda90 0x7f6c440ce540 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.501+0000 7f6c4af27700 1 -- 172.21.15.92:0/1915319343 >> 172.21.15.92:0/1915319343 conn(0x7f6c4406e4d0 msgr2=0x7f6c4406e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:54.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.501+0000 7f6c4af27700 1 -- 172.21.15.92:0/1915319343 shutdown_connections 2023-06-09T04:35:54.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.501+0000 7f6c4af27700 1 -- 172.21.15.92:0/1915319343 wait complete. 2023-06-09T04:35:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.502+0000 7f6c4af27700 1 Processor -- start 2023-06-09T04:35:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.502+0000 7f6c4af27700 1 -- start start 2023-06-09T04:35:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c4af27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c440b60d0 0x7f6c441f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c4af27700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c440cda90 0x7f6c440b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c4af27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440d03b0 0x7f6c44152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c4af27700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44153d60 con 0x7f6c440d03b0 2023-06-09T04:35:54.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c4af27700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c441f4960 con 0x7f6c440b60d0 2023-06-09T04:35:54.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c4af27700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44153fb0 con 0x7f6c440cda90 2023-06-09T04:35:54.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c494c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440d03b0 0x7f6c44152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c494c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440d03b0 0x7f6c44152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38920/0 (socket says 172.21.15.92:38920) 2023-06-09T04:35:54.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c494c4700 1 -- 172.21.15.92:0/15200745 learned_addr learned my addr 172.21.15.92:0/15200745 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c43fff700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c440cda90 0x7f6c440b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c494c4700 1 -- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c440cda90 msgr2=0x7f6c440b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c48cc3700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c440b60d0 0x7f6c441f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c494c4700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c440cda90 0x7f6c440b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.503+0000 7f6c494c4700 1 -- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c440b60d0 msgr2=0x7f6c441f4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c494c4700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c440b60d0 0x7f6c441f4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c494c4700 1 -- 172.21.15.92:0/15200745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c009a70 con 0x7f6c440d03b0 2023-06-09T04:35:54.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c494c4700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440d03b0 0x7f6c44152a90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6c4406fa30 tx=0x7f6c3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c417fa700 1 -- 172.21.15.92:0/15200745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c012630 con 0x7f6c440d03b0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c417fa700 1 -- 172.21.15.92:0/15200745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c0127b0 con 0x7f6c440d03b0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c44154290 con 0x7f6c440d03b0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c417fa700 1 -- 172.21.15.92:0/15200745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c012b60 con 0x7f6c440d03b0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.504+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c44154850 con 0x7f6c440d03b0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.505+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4404e8e0 con 0x7f6c440d03b0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.505+0000 7f6c417fa700 1 -- 172.21.15.92:0/15200745 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c3c012cc0 con 0x7f6c440d03b0 2023-06-09T04:35:54.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.506+0000 7f6c417fa700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c2806ace0 0x7f6c2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.506+0000 7f6c417fa700 1 -- 172.21.15.92:0/15200745 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c3c08be40 con 0x7f6c440d03b0 2023-06-09T04:35:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.506+0000 7f6c48cc3700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c2806ace0 0x7f6c2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.506+0000 7f6c48cc3700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c2806ace0 0x7f6c2806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000f60 tx=0x7f6c38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.510+0000 7f6c417fa700 1 -- 172.21.15.92:0/15200745 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6c3c056f80 con 0x7f6c440d03b0 2023-06-09T04:35:54.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.695+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c44070890 con 0x7f6c440d03b0 2023-06-09T04:35:54.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.696+0000 7f6c417fa700 1 -- 172.21.15.92:0/15200745 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c3c05ee60 con 0x7f6c440d03b0 2023-06-09T04:35:54.697 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c2806ace0 msgr2=0x7f6c2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c2806ace0 0x7f6c2806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000f60 tx=0x7f6c38009040 comp rx=0 tx=0).stop 2023-06-09T04:35:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440d03b0 msgr2=0x7f6c44152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440d03b0 0x7f6c44152a90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6c4406fa30 tx=0x7f6c3c004ad0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 shutdown_connections 2023-06-09T04:35:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6c2806ace0 0x7f6c2806d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c440d03b0 0x7f6c44152a90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c440b60d0 0x7f6c441f4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 --2- 172.21.15.92:0/15200745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6c440cda90 0x7f6c440b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 >> 172.21.15.92:0/15200745 conn(0x7f6c4406e4d0 msgr2=0x7f6c440b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 shutdown_connections 2023-06-09T04:35:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.698+0000 7f6c4af27700 1 -- 172.21.15.92:0/15200745 wait complete. 2023-06-09T04:35:54.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:54.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.832+0000 7ff290448700 1 Processor -- start 2023-06-09T04:35:54.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.832+0000 7ff290448700 1 -- start start 2023-06-09T04:35:54.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff290448700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff2880aed50 0x7ff2880713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff290448700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 0x7ff28815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff290448700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2880ce190 con 0x7ff2880ce3d0 2023-06-09T04:35:54.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff290448700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff288072350 con 0x7ff2880aed50 2023-06-09T04:35:54.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff290448700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff288071ab0 con 0x7ff28815d470 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28d9e3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff2880aed50 0x7ff2880713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28e1e4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff2880ce3d0 0x7ff2880ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37256/0 (socket says 172.21.15.92:37256) 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28e1e4700 1 -- 172.21.15.92:0/721143138 learned_addr learned my addr 172.21.15.92:0/721143138 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28d9e3700 1 -- 172.21.15.92:0/721143138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 msgr2=0x7ff28815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28d9e3700 1 --2- 172.21.15.92:0/721143138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 0x7ff28815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28d9e3700 1 -- 172.21.15.92:0/721143138 >> v1:172.21.15.92:6789/0 conn(0x7ff2880ce3d0 legacy=0x7ff2880ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28d9e3700 1 -- 172.21.15.92:0/721143138 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288071d90 con 0x7ff2880aed50 2023-06-09T04:35:54.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.833+0000 7ff28d1e2700 1 -- 172.21.15.92:0/721143138 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437979602 0 0) 0x7ff2880ce190 con 0x7ff2880ce3d0 2023-06-09T04:35:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff28d9e3700 1 --2- 172.21.15.92:0/721143138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880aed50 0x7ff2880713b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff284009840 tx=0x7ff284009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b529f206d8b1a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff28d1e2700 1 -- 172.21.15.92:0/721143138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011830 con 0x7ff2880aed50 2023-06-09T04:35:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff28d1e2700 1 -- 172.21.15.92:0/721143138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284011990 con 0x7ff2880aed50 2023-06-09T04:35:54.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff28d1e2700 1 -- 172.21.15.92:0/721143138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011d20 con 0x7ff2880aed50 2023-06-09T04:35:54.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff290448700 1 -- 172.21.15.92:0/721143138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880aed50 msgr2=0x7ff2880713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff290448700 1 --2- 172.21.15.92:0/721143138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880aed50 0x7ff2880713b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff284009840 tx=0x7ff284009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:54.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff290448700 1 -- 172.21.15.92:0/721143138 shutdown_connections 2023-06-09T04:35:54.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff290448700 1 --2- 172.21.15.92:0/721143138 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 0x7ff28815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff290448700 1 --2- 172.21.15.92:0/721143138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880aed50 0x7ff2880713b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.834+0000 7ff290448700 1 -- 172.21.15.92:0/721143138 >> 172.21.15.92:0/721143138 conn(0x7ff2880cc380 msgr2=0x7ff2880cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:54.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.835+0000 7ff290448700 1 -- 172.21.15.92:0/721143138 shutdown_connections 2023-06-09T04:35:54.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.835+0000 7ff290448700 1 -- 172.21.15.92:0/721143138 wait complete. 2023-06-09T04:35:54.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.836+0000 7ff290448700 1 Processor -- start 2023-06-09T04:35:54.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.836+0000 7ff290448700 1 -- start start 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff290448700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2880aed50 0x7ff288147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff290448700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880ce3d0 0x7ff2880b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff290448700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 0x7ff2880b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff290448700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881484d0 con 0x7ff2880ce3d0 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff290448700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881e4000 con 0x7ff2880aed50 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff290448700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff288155350 con 0x7ff28815d470 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880ce3d0 0x7ff2880b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880ce3d0 0x7ff2880b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38932/0 (socket says 172.21.15.92:38932) 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 -- 172.21.15.92:0/675078926 learned_addr learned my addr 172.21.15.92:0/675078926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28e1e4700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2880aed50 0x7ff288147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 -- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 msgr2=0x7ff2880b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28e9e5700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 0x7ff2880b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 0x7ff2880b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 -- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2880aed50 msgr2=0x7ff288147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2880aed50 0x7ff288147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:54.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.837+0000 7ff28d9e3700 1 -- 172.21.15.92:0/675078926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840094a0 con 0x7ff2880ce3d0 2023-06-09T04:35:54.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.838+0000 7ff28d9e3700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880ce3d0 0x7ff2880b3780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff284000c00 tx=0x7ff284004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.838+0000 7ff27effd700 1 -- 172.21.15.92:0/675078926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2840044e0 con 0x7ff2880ce3d0 2023-06-09T04:35:54.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.838+0000 7ff27effd700 1 -- 172.21.15.92:0/675078926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2840036a0 con 0x7ff2880ce3d0 2023-06-09T04:35:54.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.838+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2881e33f0 con 0x7ff2880ce3d0 2023-06-09T04:35:54.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.838+0000 7ff27effd700 1 -- 172.21.15.92:0/675078926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284003a30 con 0x7ff2880ce3d0 2023-06-09T04:35:54.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.838+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881e39b0 con 0x7ff2880ce3d0 2023-06-09T04:35:54.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.839+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2881e41b0 con 0x7ff2880ce3d0 2023-06-09T04:35:54.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.839+0000 7ff27effd700 1 -- 172.21.15.92:0/675078926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff284023070 con 0x7ff2880ce3d0 2023-06-09T04:35:54.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.840+0000 7ff27effd700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff26c06ace0 0x7ff26c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:54.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.840+0000 7ff27effd700 1 -- 172.21.15.92:0/675078926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff28408c740 con 0x7ff2880ce3d0 2023-06-09T04:35:54.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.840+0000 7ff28e1e4700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff26c06ace0 0x7ff26c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:54.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.840+0000 7ff28e1e4700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff26c06ace0 0x7ff26c06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff278000f60 tx=0x7ff278009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:54.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:54.844+0000 7ff27effd700 1 -- 172.21.15.92:0/675078926 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff284057960 con 0x7ff2880ce3d0 2023-06-09T04:35:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.029+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2881e41b0 con 0x7ff2880ce3d0 2023-06-09T04:35:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.030+0000 7ff27effd700 1 -- 172.21.15.92:0/675078926 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2881e41b0 con 0x7ff2880ce3d0 2023-06-09T04:35:55.032 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:55.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff26c06ace0 msgr2=0x7ff26c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff26c06ace0 0x7ff26c06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff278000f60 tx=0x7ff278009040 comp rx=0 tx=0).stop 2023-06-09T04:35:55.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880ce3d0 msgr2=0x7ff2880b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880ce3d0 0x7ff2880b3780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff284000c00 tx=0x7ff284004750 comp rx=0 tx=0).stop 2023-06-09T04:35:55.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 shutdown_connections 2023-06-09T04:35:55.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff26c06ace0 0x7ff26c06d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff28815d470 0x7ff2880b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2880aed50 0x7ff288147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 --2- 172.21.15.92:0/675078926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2880ce3d0 0x7ff2880b3780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 >> 172.21.15.92:0/675078926 conn(0x7ff2880cc380 msgr2=0x7ff2880afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:55.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 shutdown_connections 2023-06-09T04:35:55.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.032+0000 7ff290448700 1 -- 172.21.15.92:0/675078926 wait complete. 2023-06-09T04:35:55.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-09T04:35:55.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.165+0000 7fa7c0880700 1 Processor -- start 2023-06-09T04:35:55.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.165+0000 7fa7c0880700 1 -- start start 2023-06-09T04:35:55.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7c0880700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7c0880700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa7bc0ac400 0x7fa7bc06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7c0880700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0ce730 con 0x7fa7bc15c7d0 2023-06-09T04:35:55.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7c0880700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc06aaf0 con 0x7fa7bc0ac400 2023-06-09T04:35:55.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7c0880700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc15cba0 con 0x7fa7bc0cf010 2023-06-09T04:35:55.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7ba59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa7bc15c7d0 0x7fa7bc0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37270/0 (socket says 172.21.15.92:37270) 2023-06-09T04:35:55.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7ba59c700 1 -- 172.21.15.92:0/760916014 learned_addr learned my addr 172.21.15.92:0/760916014 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:55.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7b9d9b700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7bad9d700 1 --2- 172.21.15.92:0/760916014 >> v2:172.21.15.123:3300/0 conn(0x7fa7bc0ac400 0x7fa7bc06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773157284 0 0) 0x7fa7b0001290 con 0x7fa7bc15c7d0 2023-06-09T04:35:55.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.166+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a40035b0 con 0x7fa7bc15c7d0 2023-06-09T04:35:55.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2488934922 0 0) 0x7fa7a40035b0 con 0x7fa7bc15c7d0 2023-06-09T04:35:55.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 msgr2=0x7fa7bc0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:55.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 >> v2:172.21.15.123:3300/0 conn(0x7fa7bc0ac400 msgr2=0x7fa7bc06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:55.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 --2- 172.21.15.92:0/760916014 >> v2:172.21.15.123:3300/0 conn(0x7fa7bc0ac400 0x7fa7bc06e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc069ee0 con 0x7fa7bc15c7d0 2023-06-09T04:35:55.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fa7b0001980 con 0x7fa7bc15c7d0 2023-06-09T04:35:55.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 >> v1:172.21.15.92:6789/0 conn(0x7fa7bc15c7d0 legacy=0x7fa7bc0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4004d90 0x7fa7a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc06aaf0 con 0x7fa7a4003a00 2023-06-09T04:35:55.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc15cba0 con 0x7fa7a4004d90 2023-06-09T04:35:55.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400ba10 con 0x7fa7bc0cf010 2023-06-09T04:35:55.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7ba59c700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7bad9d700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4004d90 0x7fa7a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b9d9b700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7ba59c700 1 -- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 msgr2=0x7fa7a4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7ba59c700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7a4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7ba59c700 1 -- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4004d90 msgr2=0x7fa7a4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7ba59c700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4004d90 0x7fa7a4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7ba59c700 1 -- 172.21.15.92:0/760916014 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40035b0 con 0x7fa7a4003a00 2023-06-09T04:35:55.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b9d9b700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7a4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:55.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7ba59c700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000c210 tx=0x7fa7a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9387bcbe1ad42bcb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:55.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0007dc0 con 0x7fa7a4003a00 2023-06-09T04:35:55.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b000cdb0 con 0x7fa7a4003a00 2023-06-09T04:35:55.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.167+0000 7fa7b959a700 1 -- 172.21.15.92:0/760916014 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0007310 con 0x7fa7a4003a00 2023-06-09T04:35:55.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.168+0000 7fa7c0880700 1 -- 172.21.15.92:0/760916014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7a4003a00 msgr2=0x7fa7a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.168+0000 7fa7c0880700 1 --2- 172.21.15.92:0/760916014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000c210 tx=0x7fa7a4005880 comp rx=0 tx=0).stop 2023-06-09T04:35:55.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.168+0000 7fa7bad9d700 1 -- 172.21.15.92:0/760916014 reap_dead start 2023-06-09T04:35:55.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.168+0000 7fa7c0880700 1 -- 172.21.15.92:0/760916014 shutdown_connections 2023-06-09T04:35:55.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.168+0000 7fa7c0880700 1 -- 172.21.15.92:0/760916014 >> 172.21.15.92:0/760916014 conn(0x7fa7bc098c10 msgr2=0x7fa7bc0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:55.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.168+0000 7fa7c0880700 1 -- 172.21.15.92:0/760916014 shutdown_connections 2023-06-09T04:35:55.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.168+0000 7fa7c0880700 1 -- 172.21.15.92:0/760916014 wait complete. 2023-06-09T04:35:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.170+0000 7fa7c0880700 1 Processor -- start 2023-06-09T04:35:55.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.170+0000 7fa7c0880700 1 -- start start 2023-06-09T04:35:55.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7c0880700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4003a00 0x7fa7bc1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7c0880700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0b2710 0x7fa7bc0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7c0880700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0b4b90 0x7fa7bc0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7c0880700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c5e20 con 0x7fa7bc0b2710 2023-06-09T04:35:55.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7c0880700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b5550 con 0x7fa7a4003a00 2023-06-09T04:35:55.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7c0880700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b4270 con 0x7fa7bc0b4b90 2023-06-09T04:35:55.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0b2710 0x7fa7bc0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0b2710 0x7fa7bc0b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38952/0 (socket says 172.21.15.92:38952) 2023-06-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 -- 172.21.15.92:0/2932984193 learned_addr learned my addr 172.21.15.92:0/2932984193 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7bad9d700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0b4b90 0x7fa7bc0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7ba59c700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4003a00 0x7fa7bc1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 -- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0b4b90 msgr2=0x7fa7bc0b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0b4b90 0x7fa7bc0b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 -- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4003a00 msgr2=0x7fa7bc1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4003a00 0x7fa7bc1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 -- 172.21.15.92:0/2932984193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0b31d0 con 0x7fa7bc0b2710 2023-06-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7b9d9b700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0b2710 0x7fa7bc0b2b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac010870 tx=0x7fa7ac010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.171+0000 7fa7aaffd700 1 -- 172.21.15.92:0/2932984193 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac010e70 con 0x7fa7bc0b2710 2023-06-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.172+0000 7fa7aaffd700 1 -- 172.21.15.92:0/2932984193 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac004020 con 0x7fa7bc0b2710 2023-06-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.172+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0b34b0 con 0x7fa7bc0b2710 2023-06-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.172+0000 7fa7aaffd700 1 -- 172.21.15.92:0/2932984193 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac0043b0 con 0x7fa7bc0b2710 2023-06-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.172+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc0b3a70 con 0x7fa7bc0b2710 2023-06-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.172+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc066d50 con 0x7fa7bc0b2710 2023-06-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.173+0000 7fa7aaffd700 1 -- 172.21.15.92:0/2932984193 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7ac00e8d0 con 0x7fa7bc0b2710 2023-06-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.173+0000 7fa7aaffd700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa79c06ace0 0x7fa79c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.173+0000 7fa7aaffd700 1 -- 172.21.15.92:0/2932984193 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7ac0894d0 con 0x7fa7bc0b2710 2023-06-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.173+0000 7fa7ba59c700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa79c06ace0 0x7fa79c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.174+0000 7fa7ba59c700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa79c06ace0 0x7fa79c06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0c4e90 tx=0x7fa7b000a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.178+0000 7fa7aaffd700 1 -- 172.21.15.92:0/2932984193 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7ac058010 con 0x7fa7bc0b2710 2023-06-09T04:35:55.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.384+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa7bc0740e0 con 0x7fa7bc0b2710 2023-06-09T04:35:55.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.384+0000 7fa7aaffd700 1 -- 172.21.15.92:0/2932984193 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa7ac057ba0 con 0x7fa7bc0b2710 2023-06-09T04:35:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa79c06ace0 msgr2=0x7fa79c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa79c06ace0 0x7fa79c06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0c4e90 tx=0x7fa7b000a000 comp rx=0 tx=0).stop 2023-06-09T04:35:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0b2710 msgr2=0x7fa7bc0b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0b2710 0x7fa7bc0b2b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac010870 tx=0x7fa7ac010c30 comp rx=0 tx=0).stop 2023-06-09T04:35:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 shutdown_connections 2023-06-09T04:35:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa79c06ace0 0x7fa79c06d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0b4b90 0x7fa7bc0b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7a4003a00 0x7fa7bc1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 --2- 172.21.15.92:0/2932984193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0b2710 0x7fa7bc0b2b80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 >> 172.21.15.92:0/2932984193 conn(0x7fa7bc098c10 msgr2=0x7fa7bc0c8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:55.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 shutdown_connections 2023-06-09T04:35:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.387+0000 7fa7c0880700 1 -- 172.21.15.92:0/2932984193 wait complete. 2023-06-09T04:35:55.390 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-06-09T04:35:55.400 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-06-09T04:35:55.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:55.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-06-09T04:35:55.438 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:35:55.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:55.439 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-09T04:35:55.496 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:55.496 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:55.517 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:55.518 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:55.575 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:35:55.576 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:35:55.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.753+0000 7fcc6efbd700 1 Processor -- start 2023-06-09T04:35:55.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.753+0000 7fcc6efbd700 1 -- start start 2023-06-09T04:35:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.753+0000 7fcc6efbd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc6815c7d0 0x7fcc680ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.753+0000 7fcc6efbd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680ce730 con 0x7fcc680cf010 2023-06-09T04:35:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.753+0000 7fcc6efbd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6806aaf0 con 0x7fcc680ac400 2023-06-09T04:35:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.753+0000 7fcc6efbd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815cba0 con 0x7fcc6815c7d0 2023-06-09T04:35:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc6d55a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcc680cf010 0x7fcc6806e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37294/0 (socket says 172.21.15.92:37294) 2023-06-09T04:35:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc6d55a700 1 -- 172.21.15.92:0/1155760048 learned_addr learned my addr 172.21.15.92:0/1155760048 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc6cd59700 1 --2- 172.21.15.92:0/1155760048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc6815c7d0 0x7fcc680ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc677fe700 1 -- 172.21.15.92:0/1155760048 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821574164 0 0) 0x7fcc60001490 con 0x7fcc680cf010 2023-06-09T04:35:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc677fe700 1 -- 172.21.15.92:0/1155760048 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc580035b0 con 0x7fcc680cf010 2023-06-09T04:35:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc677fe700 1 -- 172.21.15.92:0/1155760048 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815433437 0 0) 0x7fcc6806aaf0 con 0x7fcc680ac400 2023-06-09T04:35:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc6cd59700 1 -- 172.21.15.92:0/1155760048 >> v1:172.21.15.123:6789/0 conn(0x7fcc680ac400 legacy=0x7fcc680ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc6cd59700 1 -- 172.21.15.92:0/1155760048 >> v1:172.21.15.92:6789/0 conn(0x7fcc680cf010 legacy=0x7fcc6806e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc6cd59700 1 -- 172.21.15.92:0/1155760048 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68069ee0 con 0x7fcc6815c7d0 2023-06-09T04:35:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc6cd59700 1 --2- 172.21.15.92:0/1155760048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc6815c7d0 0x7fcc680ce1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009850 tx=0x7fcc5c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e876dad8047da65 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc677fe700 1 -- 172.21.15.92:0/1155760048 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60001b80 con 0x7fcc6815c7d0 2023-06-09T04:35:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc677fe700 1 -- 172.21.15.92:0/1155760048 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c005c30 con 0x7fcc6815c7d0 2023-06-09T04:35:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.754+0000 7fcc677fe700 1 -- 172.21.15.92:0/1155760048 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c006030 con 0x7fcc6815c7d0 2023-06-09T04:35:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.755+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1155760048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc6815c7d0 msgr2=0x7fcc680ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.755+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1155760048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc6815c7d0 0x7fcc680ce1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009850 tx=0x7fcc5c009b60 comp rx=0 tx=0).stop 2023-06-09T04:35:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.755+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1155760048 shutdown_connections 2023-06-09T04:35:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.755+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1155760048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc6815c7d0 0x7fcc680ce1f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.755+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1155760048 >> 172.21.15.92:0/1155760048 conn(0x7fcc68098c10 msgr2=0x7fcc680cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.755+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1155760048 shutdown_connections 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.755+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1155760048 wait complete. 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.756+0000 7fcc6efbd700 1 Processor -- start 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.756+0000 7fcc6efbd700 1 -- start start 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6efbd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc680ac400 0x7fcc68148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6efbd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc680cf010 0x7fcc68145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6efbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc6815c7d0 0x7fcc680c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6efbd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680cabc0 con 0x7fcc6815c7d0 2023-06-09T04:35:55.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6efbd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680ca8b0 con 0x7fcc680cf010 2023-06-09T04:35:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6efbd700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68145dd0 con 0x7fcc680ac400 2023-06-09T04:35:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc6815c7d0 0x7fcc680c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc6815c7d0 0x7fcc680c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38966/0 (socket says 172.21.15.92:38966) 2023-06-09T04:35:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 -- 172.21.15.92:0/1195383288 learned_addr learned my addr 172.21.15.92:0/1195383288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6cd59700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc680ac400 0x7fcc68148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc67fff700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc680cf010 0x7fcc68145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 -- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc680ac400 msgr2=0x7fcc68148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc680ac400 0x7fcc68148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 -- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc680cf010 msgr2=0x7fcc68145890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc680cf010 0x7fcc68145890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.757+0000 7fcc6d55a700 1 -- 172.21.15.92:0/1195383288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c009430 con 0x7fcc6815c7d0 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.758+0000 7fcc6d55a700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc6815c7d0 0x7fcc680c5420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcc60001530 tx=0x7fcc6000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.758+0000 7fcc657fa700 1 -- 172.21.15.92:0/1195383288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000acd0 con 0x7fcc6815c7d0 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.758+0000 7fcc657fa700 1 -- 172.21.15.92:0/1195383288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6000ae30 con 0x7fcc6815c7d0 2023-06-09T04:35:55.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.758+0000 7fcc657fa700 1 -- 172.21.15.92:0/1195383288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60007310 con 0x7fcc6815c7d0 2023-06-09T04:35:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.758+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680c5d70 con 0x7fcc6815c7d0 2023-06-09T04:35:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.758+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc680c6330 con 0x7fcc6815c7d0 2023-06-09T04:35:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.759+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc680c6550 con 0x7fcc6815c7d0 2023-06-09T04:35:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.759+0000 7fcc657fa700 1 -- 172.21.15.92:0/1195383288 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc60007470 con 0x7fcc6815c7d0 2023-06-09T04:35:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.760+0000 7fcc657fa700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcc4c06ace0 0x7fcc4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.760+0000 7fcc657fa700 1 -- 172.21.15.92:0/1195383288 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc60088f00 con 0x7fcc6815c7d0 2023-06-09T04:35:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.760+0000 7fcc6cd59700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcc4c06ace0 0x7fcc4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.760+0000 7fcc6cd59700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcc4c06ace0 0x7fcc4c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c000c00 tx=0x7fcc5c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:55.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.764+0000 7fcc657fa700 1 -- 172.21.15.92:0/1195383288 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcc60053fb0 con 0x7fcc6815c7d0 2023-06-09T04:35:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.954+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc680cad00 con 0x7fcc6815c7d0 2023-06-09T04:35:55.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.955+0000 7fcc657fa700 1 -- 172.21.15.92:0/1195383288 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18255 (secure 0 0 0) 0x7fcc680cad00 con 0x7fcc6815c7d0 2023-06-09T04:35:55.957 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:55.957 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":24,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:18.414014+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcc4c06ace0 msgr2=0x7fcc4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcc4c06ace0 0x7fcc4c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c000c00 tx=0x7fcc5c007000 comp rx=0 tx=0).stop 2023-06-09T04:35:55.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc6815c7d0 msgr2=0x7fcc680c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc6815c7d0 0x7fcc680c5420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcc60001530 tx=0x7fcc6000a340 comp rx=0 tx=0).stop 2023-06-09T04:35:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 shutdown_connections 2023-06-09T04:35:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fcc4c06ace0 0x7fcc4c06d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc6815c7d0 0x7fcc680c5420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcc680ac400 0x7fcc68148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 --2- 172.21.15.92:0/1195383288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc680cf010 0x7fcc68145890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:55.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 >> 172.21.15.92:0/1195383288 conn(0x7fcc68098c10 msgr2=0x7fcc6806e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 shutdown_connections 2023-06-09T04:35:55.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:55.958+0000 7fcc6efbd700 1 -- 172.21.15.92:0/1195383288 wait complete. 2023-06-09T04:35:55.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:55.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:56.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.093+0000 7f65255d7700 1 Processor -- start 2023-06-09T04:35:56.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.093+0000 7f65255d7700 1 -- start start 2023-06-09T04:35:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f65255d7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200cf100 0x7f65200d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f65255d7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65200d3800 0x7f65200cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f65255d7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65200d2b00 con 0x7f652015ced0 2023-06-09T04:35:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f65255d7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65200cd3c0 con 0x7f65200d3800 2023-06-09T04:35:56.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f65255d7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cd6a0 con 0x7f65200cf100 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f652015ced0 0x7f652015d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37304/0 (socket says 172.21.15.92:37304) 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651effd700 1 -- 172.21.15.92:0/2927559207 learned_addr learned my addr 172.21.15.92:0/2927559207 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651f7fe700 1 --2- 172.21.15.92:0/2927559207 >> v2:172.21.15.92:3300/0 conn(0x7f65200d3800 0x7f65200cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651e7fc700 1 --2- 172.21.15.92:0/2927559207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200cf100 0x7f65200d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651f7fe700 1 -- 172.21.15.92:0/2927559207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200cf100 msgr2=0x7f65200d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651f7fe700 1 --2- 172.21.15.92:0/2927559207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200cf100 0x7f65200d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651f7fe700 1 -- 172.21.15.92:0/2927559207 >> v1:172.21.15.92:6789/0 conn(0x7f652015ced0 legacy=0x7f652015d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651dffb700 1 -- 172.21.15.92:0/2927559207 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091670344 0 0) 0x7f65200d2b00 con 0x7f652015ced0 2023-06-09T04:35:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651f7fe700 1 -- 172.21.15.92:0/2927559207 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200d2da0 con 0x7f65200d3800 2023-06-09T04:35:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.094+0000 7f651f7fe700 1 --2- 172.21.15.92:0/2927559207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200d3800 0x7f65200cf670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6518009d90 tx=0x7f651800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec22566fd42c10ba server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f651dffb700 1 -- 172.21.15.92:0/2927559207 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800e070 con 0x7f65200d3800 2023-06-09T04:35:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f651dffb700 1 -- 172.21.15.92:0/2927559207 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651800bb20 con 0x7f65200d3800 2023-06-09T04:35:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f651dffb700 1 -- 172.21.15.92:0/2927559207 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518012430 con 0x7f65200d3800 2023-06-09T04:35:56.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 -- 172.21.15.92:0/2927559207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200d3800 msgr2=0x7f65200cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 --2- 172.21.15.92:0/2927559207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200d3800 0x7f65200cf670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6518009d90 tx=0x7f651800b330 comp rx=0 tx=0).stop 2023-06-09T04:35:56.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 -- 172.21.15.92:0/2927559207 shutdown_connections 2023-06-09T04:35:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 --2- 172.21.15.92:0/2927559207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200d3800 0x7f65200cf670 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 --2- 172.21.15.92:0/2927559207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65200cf100 0x7f65200d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 -- 172.21.15.92:0/2927559207 >> 172.21.15.92:0/2927559207 conn(0x7f652006b2e0 msgr2=0x7f652006b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 -- 172.21.15.92:0/2927559207 shutdown_connections 2023-06-09T04:35:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.095+0000 7f65255d7700 1 -- 172.21.15.92:0/2927559207 wait complete. 2023-06-09T04:35:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 Processor -- start 2023-06-09T04:35:56.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 -- start start 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200cf100 0x7f6520155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d3800 0x7f65200c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015ced0 0x7f65200c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6520143c70 con 0x7f65200cf100 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6520156280 con 0x7f65200d3800 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f65255d7700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6520156560 con 0x7f652015ced0 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f651effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200cf100 0x7f6520155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f651effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200cf100 0x7f6520155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38978/0 (socket says 172.21.15.92:38978) 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.097+0000 7f651effd700 1 -- 172.21.15.92:0/679977173 learned_addr learned my addr 172.21.15.92:0/679977173 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651e7fc700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d3800 0x7f65200c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651f7fe700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015ced0 0x7f65200c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651effd700 1 -- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015ced0 msgr2=0x7f65200c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651effd700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015ced0 0x7f65200c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651effd700 1 -- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d3800 msgr2=0x7f65200c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651effd700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d3800 0x7f65200c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651effd700 1 -- 172.21.15.92:0/679977173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6518009a70 con 0x7f65200cf100 2023-06-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f651effd700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200cf100 0x7f6520155a30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f651400c550 tx=0x7f651400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f65077fe700 1 -- 172.21.15.92:0/679977173 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514004c80 con 0x7f65200cf100 2023-06-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f65077fe700 1 -- 172.21.15.92:0/679977173 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6514004e00 con 0x7f65200cf100 2023-06-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f65077fe700 1 -- 172.21.15.92:0/679977173 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514007310 con 0x7f65200cf100 2023-06-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6520156840 con 0x7f65200cf100 2023-06-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.098+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6520155f70 con 0x7f65200cf100 2023-06-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.099+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652004e8e0 con 0x7f65200cf100 2023-06-09T04:35:56.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.099+0000 7f65077fe700 1 -- 172.21.15.92:0/679977173 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6514007470 con 0x7f65200cf100 2023-06-09T04:35:56.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.100+0000 7f65077fe700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f650c06ace0 0x7f650c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.100+0000 7f65077fe700 1 -- 172.21.15.92:0/679977173 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65140891f0 con 0x7f65200cf100 2023-06-09T04:35:56.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.100+0000 7f651e7fc700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f650c06ace0 0x7f650c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.100+0000 7f651e7fc700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f650c06ace0 0x7f650c06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65200c8390 tx=0x7f6510008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.105+0000 7f65077fe700 1 -- 172.21.15.92:0/679977173 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65140542a0 con 0x7f65200cf100 2023-06-09T04:35:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.294+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65200b50c0 con 0x7f65200cf100 2023-06-09T04:35:56.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.295+0000 7f65077fe700 1 -- 172.21.15.92:0/679977173 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65140578c0 con 0x7f65200cf100 2023-06-09T04:35:56.295 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f650c06ace0 msgr2=0x7f650c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f650c06ace0 0x7f650c06d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65200c8390 tx=0x7f6510008040 comp rx=0 tx=0).stop 2023-06-09T04:35:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200cf100 msgr2=0x7f6520155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200cf100 0x7f6520155a30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f651400c550 tx=0x7f651400c910 comp rx=0 tx=0).stop 2023-06-09T04:35:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 shutdown_connections 2023-06-09T04:35:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f650c06ace0 0x7f650c06d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f652015ced0 0x7f65200c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65200cf100 0x7f6520155a30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 --2- 172.21.15.92:0/679977173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65200d3800 0x7f65200c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 >> 172.21.15.92:0/679977173 conn(0x7f652006b2e0 msgr2=0x7f652015f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 shutdown_connections 2023-06-09T04:35:56.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.297+0000 7f65255d7700 1 -- 172.21.15.92:0/679977173 wait complete. 2023-06-09T04:35:56.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:56.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.432+0000 7fe506247700 1 Processor -- start 2023-06-09T04:35:56.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.432+0000 7fe506247700 1 -- start start 2023-06-09T04:35:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe506247700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000d03b0 0x7fe5000cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe506247700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe5000cd8c0 0x7fe5000ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe506247700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5000cb700 con 0x7fe5000b60d0 2023-06-09T04:35:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe506247700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5000cdd40 con 0x7fe5000cd8c0 2023-06-09T04:35:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe506247700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000b64a0 con 0x7fe5000d03b0 2023-06-09T04:35:56.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4feffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000d03b0 0x7fe5000cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4fffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe5000cd8c0 0x7fe5000ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4feffd700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000d03b0 0x7fe5000cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:47898/0 (socket says 172.21.15.92:47898) 2023-06-09T04:35:56.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4feffd700 1 -- 172.21.15.92:0/1561774113 learned_addr learned my addr 172.21.15.92:0/1561774113 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:56.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4ff7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe5000b60d0 0x7fe50015e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:59814/0 (socket says 172.21.15.92:59814) 2023-06-09T04:35:56.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4feffd700 1 -- 172.21.15.92:0/1561774113 >> v2:172.21.15.123:3300/0 conn(0x7fe5000cd8c0 msgr2=0x7fe5000ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:56.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4feffd700 1 --2- 172.21.15.92:0/1561774113 >> v2:172.21.15.123:3300/0 conn(0x7fe5000cd8c0 0x7fe5000ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4feffd700 1 -- 172.21.15.92:0/1561774113 >> v1:172.21.15.123:6789/0 conn(0x7fe5000b60d0 legacy=0x7fe50015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.433+0000 7fe4feffd700 1 -- 172.21.15.92:0/1561774113 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000ccf50 con 0x7fe5000d03b0 2023-06-09T04:35:56.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe4feffd700 1 --2- 172.21.15.92:0/1561774113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000d03b0 0x7fe5000cd380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00098f0 tx=0x7fe4f0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3740f9f138af9049 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe4fe7fc700 1 -- 172.21.15.92:0/1561774113 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0011830 con 0x7fe5000d03b0 2023-06-09T04:35:56.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe4fe7fc700 1 -- 172.21.15.92:0/1561774113 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0011990 con 0x7fe5000d03b0 2023-06-09T04:35:56.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe4fe7fc700 1 -- 172.21.15.92:0/1561774113 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0011d20 con 0x7fe5000d03b0 2023-06-09T04:35:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 -- 172.21.15.92:0/1561774113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000d03b0 msgr2=0x7fe5000cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 --2- 172.21.15.92:0/1561774113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000d03b0 0x7fe5000cd380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00098f0 tx=0x7fe4f0009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 -- 172.21.15.92:0/1561774113 shutdown_connections 2023-06-09T04:35:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 --2- 172.21.15.92:0/1561774113 >> v2:172.21.15.123:3300/0 conn(0x7fe5000cd8c0 0x7fe5000ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 --2- 172.21.15.92:0/1561774113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000d03b0 0x7fe5000cd380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 -- 172.21.15.92:0/1561774113 >> 172.21.15.92:0/1561774113 conn(0x7fe50006e4d0 msgr2=0x7fe50006e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 -- 172.21.15.92:0/1561774113 shutdown_connections 2023-06-09T04:35:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.434+0000 7fe506247700 1 -- 172.21.15.92:0/1561774113 wait complete. 2023-06-09T04:35:56.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 Processor -- start 2023-06-09T04:35:56.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 -- start start 2023-06-09T04:35:56.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5000b60d0 0x7fe5001e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000cd8c0 0x7fe5001459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5000d03b0 0x7fe500145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe500146f30 con 0x7fe5000d03b0 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5001f11a0 con 0x7fe5000b60d0 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.436+0000 7fe506247700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe500146590 con 0x7fe5000cd8c0 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5000d03b0 0x7fe500145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5000d03b0 0x7fe500145f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38996/0 (socket says 172.21.15.92:38996) 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 -- 172.21.15.92:0/677735757 learned_addr learned my addr 172.21.15.92:0/677735757 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4ff7fe700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5000b60d0 0x7fe5001e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4feffd700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000cd8c0 0x7fe5001459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 -- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000cd8c0 msgr2=0x7fe5001459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000cd8c0 0x7fe5001459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 -- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5000b60d0 msgr2=0x7fe5001e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5000b60d0 0x7fe5001e3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 -- 172.21.15.92:0/677735757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f0009550 con 0x7fe5000d03b0 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4fffff700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5000d03b0 0x7fe500145f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800efc0 tx=0x7fe4f800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4e7fff700 1 -- 172.21.15.92:0/677735757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f800cad0 con 0x7fe5000d03b0 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4e7fff700 1 -- 172.21.15.92:0/677735757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f800f040 con 0x7fe5000d03b0 2023-06-09T04:35:56.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe500146870 con 0x7fe5000d03b0 2023-06-09T04:35:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe4e7fff700 1 -- 172.21.15.92:0/677735757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f800ce70 con 0x7fe5000d03b0 2023-06-09T04:35:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.437+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe500147350 con 0x7fe5000d03b0 2023-06-09T04:35:56.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.438+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50004e8e0 con 0x7fe5000d03b0 2023-06-09T04:35:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.439+0000 7fe4e7fff700 1 -- 172.21.15.92:0/677735757 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4f8014480 con 0x7fe5000d03b0 2023-06-09T04:35:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.439+0000 7fe4e7fff700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe4ec06ace0 0x7fe4ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.439+0000 7fe4e7fff700 1 -- 172.21.15.92:0/677735757 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4f8089c80 con 0x7fe5000d03b0 2023-06-09T04:35:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.439+0000 7fe4ff7fe700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe4ec06ace0 0x7fe4ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.439+0000 7fe4ff7fe700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe4ec06ace0 0x7fe4ec06d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4000f60 tx=0x7fe4f4007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.443+0000 7fe4e7fff700 1 -- 172.21.15.92:0/677735757 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe4f8054d30 con 0x7fe5000d03b0 2023-06-09T04:35:56.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.631+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe500070890 con 0x7fe5000d03b0 2023-06-09T04:35:56.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.632+0000 7fe4e7fff700 1 -- 172.21.15.92:0/677735757 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe4f80180a0 con 0x7fe5000d03b0 2023-06-09T04:35:56.632 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:56.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe4ec06ace0 msgr2=0x7fe4ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe4ec06ace0 0x7fe4ec06d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4000f60 tx=0x7fe4f4007ed0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5000d03b0 msgr2=0x7fe500145f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5000d03b0 0x7fe500145f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800efc0 tx=0x7fe4f800c0c0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 shutdown_connections 2023-06-09T04:35:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fe4ec06ace0 0x7fe4ec06d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5000d03b0 0x7fe500145f00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5000b60d0 0x7fe5001e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 --2- 172.21.15.92:0/677735757 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe5000cd8c0 0x7fe5001459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.633+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 >> 172.21.15.92:0/677735757 conn(0x7fe50006e4d0 msgr2=0x7fe5000b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.634+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 shutdown_connections 2023-06-09T04:35:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.634+0000 7fe506247700 1 -- 172.21.15.92:0/677735757 wait complete. 2023-06-09T04:35:56.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.768+0000 7f911f149700 1 Processor -- start 2023-06-09T04:35:56.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.769+0000 7f911f149700 1 -- start start 2023-06-09T04:35:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.769+0000 7f911f149700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 0x7f911815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.769+0000 7f911f149700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911815d040 con 0x7f91180cce60 2023-06-09T04:35:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.769+0000 7f911f149700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9118160ab0 con 0x7f91180cf440 2023-06-09T04:35:56.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.769+0000 7f911f149700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911815ca10 con 0x7f9118069db0 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.769+0000 7f911d6e6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f91180cce60 0x7f911806ee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37336/0 (socket says 172.21.15.92:37336) 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.769+0000 7f911d6e6700 1 -- 172.21.15.92:0/3903513566 learned_addr learned my addr 172.21.15.92:0/3903513566 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f911cee5700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 0x7f911815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791053072 0 0) 0x7f911815d040 con 0x7f91180cce60 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91080035b0 con 0x7f91180cce60 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2617988097 0 0) 0x7f91080035b0 con 0x7f91180cce60 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 msgr2=0x7f911815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:56.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 0x7f911815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 >> v1:172.21.15.123:6789/0 conn(0x7f91180cf440 legacy=0x7f911806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118160d50 con 0x7f91180cce60 2023-06-09T04:35:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f91100019f0 con 0x7f91180cce60 2023-06-09T04:35:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 >> v1:172.21.15.92:6789/0 conn(0x7f91180cce60 legacy=0x7f911806ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9108003a30 0x7f9108003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9108004850 0x7f9108004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 0x7f9108005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911815ca10 con 0x7f9108004850 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910800bb10 con 0x7f9108003a30 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910800bcd0 con 0x7f9118069db0 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f9117fff700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9108004850 0x7f9108004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f911d6e6700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 0x7f9108005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f911cee5700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9108003a30 0x7f9108003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f9117fff700 1 -- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 msgr2=0x7f9108005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:56.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f9117fff700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9118069db0 0x7f9108005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f9117fff700 1 -- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9108003a30 msgr2=0x7f9108003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f9117fff700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9108003a30 0x7f9108003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f9117fff700 1 -- 172.21.15.92:0/3903513566 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91080035b0 con 0x7f9108004850 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.770+0000 7f911cee5700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9108003a30 0x7f9108003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f9117fff700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9108004850 0x7f9108004cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f91040017c0 tx=0x7f9108005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefd4f8174eba4be server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040052d0 con 0x7f9108004850 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104005430 con 0x7f9108004850 2023-06-09T04:35:56.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f91177fe700 1 -- 172.21.15.92:0/3903513566 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040038c0 con 0x7f9108004850 2023-06-09T04:35:56.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f911f149700 1 -- 172.21.15.92:0/3903513566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9108004850 msgr2=0x7f9108004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f911f149700 1 --2- 172.21.15.92:0/3903513566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9108004850 0x7f9108004cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f91040017c0 tx=0x7f9108005880 comp rx=0 tx=0).stop 2023-06-09T04:35:56.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f911d6e6700 1 -- 172.21.15.92:0/3903513566 reap_dead start 2023-06-09T04:35:56.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f911f149700 1 -- 172.21.15.92:0/3903513566 shutdown_connections 2023-06-09T04:35:56.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f911f149700 1 -- 172.21.15.92:0/3903513566 >> 172.21.15.92:0/3903513566 conn(0x7f91180cc380 msgr2=0x7f91180cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:56.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f911f149700 1 -- 172.21.15.92:0/3903513566 shutdown_connections 2023-06-09T04:35:56.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.771+0000 7f911f149700 1 -- 172.21.15.92:0/3903513566 wait complete. 2023-06-09T04:35:56.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 Processor -- start 2023-06-09T04:35:56.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 -- start start 2023-06-09T04:35:56.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9108004850 0x7f91181550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91181558a0 0x7f9118155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91180af340 0x7f91180af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c44d0 con 0x7f91181558a0 2023-06-09T04:35:56.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c9810 con 0x7f91180af340 2023-06-09T04:35:56.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.773+0000 7f911f149700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c6b70 con 0x7f9108004850 2023-06-09T04:35:56.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91181558a0 0x7f9118155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91181558a0 0x7f9118155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39014/0 (socket says 172.21.15.92:39014) 2023-06-09T04:35:56.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 -- 172.21.15.92:0/178282767 learned_addr learned my addr 172.21.15.92:0/178282767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:56.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f911d6e6700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91180af340 0x7f91180af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f911cee5700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9108004850 0x7f91181550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 -- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9108004850 msgr2=0x7f91181550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9108004850 0x7f91181550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 -- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91180af340 msgr2=0x7f91180af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91180af340 0x7f91180af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 -- 172.21.15.92:0/178282767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180c6e50 con 0x7f91181558a0 2023-06-09T04:35:56.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f9117fff700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91181558a0 0x7f9118155d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f910800bae0 tx=0x7f9104001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f91157fa700 1 -- 172.21.15.92:0/178282767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104006ea0 con 0x7f91181558a0 2023-06-09T04:35:56.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f91157fa700 1 -- 172.21.15.92:0/178282767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104007240 con 0x7f91181558a0 2023-06-09T04:35:56.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91180c46e0 con 0x7f91181558a0 2023-06-09T04:35:56.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f91157fa700 1 -- 172.21.15.92:0/178282767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040075f0 con 0x7f91181558a0 2023-06-09T04:35:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.774+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9118155630 con 0x7f91181558a0 2023-06-09T04:35:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.775+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91180c4d30 con 0x7f91181558a0 2023-06-09T04:35:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.776+0000 7f91157fa700 1 -- 172.21.15.92:0/178282767 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9104003680 con 0x7f91181558a0 2023-06-09T04:35:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.776+0000 7f91157fa700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f90fc06ace0 0x7f90fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.776+0000 7f91157fa700 1 -- 172.21.15.92:0/178282767 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f910408a2e0 con 0x7f91181558a0 2023-06-09T04:35:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.776+0000 7f911cee5700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f90fc06ace0 0x7f90fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:56.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.777+0000 7f911cee5700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f90fc06ace0 0x7f90fc06d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f91181f58d0 tx=0x7f910c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:56.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.781+0000 7f91157fa700 1 -- 172.21.15.92:0/178282767 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91040554d0 con 0x7f91181558a0 2023-06-09T04:35:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.964+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f911806be50 con 0x7f91181558a0 2023-06-09T04:35:56.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.965+0000 7f91157fa700 1 -- 172.21.15.92:0/178282767 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9104058af0 con 0x7f91181558a0 2023-06-09T04:35:56.966 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:56.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f90fc06ace0 msgr2=0x7f90fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f90fc06ace0 0x7f90fc06d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f91181f58d0 tx=0x7f910c008040 comp rx=0 tx=0).stop 2023-06-09T04:35:56.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91181558a0 msgr2=0x7f9118155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91181558a0 0x7f9118155d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f910800bae0 tx=0x7f9104001120 comp rx=0 tx=0).stop 2023-06-09T04:35:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 shutdown_connections 2023-06-09T04:35:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f90fc06ace0 0x7f90fc06d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91180af340 0x7f91180af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9108004850 0x7f91181550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 --2- 172.21.15.92:0/178282767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91181558a0 0x7f9118155d10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 >> 172.21.15.92:0/178282767 conn(0x7f91180cc380 msgr2=0x7f911815be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 shutdown_connections 2023-06-09T04:35:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:56.967+0000 7f911f149700 1 -- 172.21.15.92:0/178282767 wait complete. 2023-06-09T04:35:56.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:57.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.103+0000 7f0963e53700 1 Processor -- start 2023-06-09T04:35:57.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.103+0000 7f0963e53700 1 -- start start 2023-06-09T04:35:57.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0963e53700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f095c0ce3d0 0x7f095c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0963e53700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f095c15d470 0x7f095c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0963e53700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095c0ce190 con 0x7f095c0aed50 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0963e53700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c072350 con 0x7f095c0ce3d0 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0963e53700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c071ab0 con 0x7f095c15d470 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0961bef700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f095c0ce3d0 0x7f095c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0961bef700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f095c0ce3d0 0x7f095c0713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39020/0 (socket says 172.21.15.92:39020) 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f09613ee700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f095c0aed50 0x7f095c0ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37352/0 (socket says 172.21.15.92:37352) 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.104+0000 7f0961bef700 1 -- 172.21.15.92:0/1204727472 learned_addr learned my addr 172.21.15.92:0/1204727472 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f09623f0700 1 --2- 172.21.15.92:0/1204727472 >> v2:172.21.15.123:3300/0 conn(0x7f095c15d470 0x7f095c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0961bef700 1 -- 172.21.15.92:0/1204727472 >> v2:172.21.15.123:3300/0 conn(0x7f095c15d470 msgr2=0x7f095c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0961bef700 1 --2- 172.21.15.92:0/1204727472 >> v2:172.21.15.123:3300/0 conn(0x7f095c15d470 0x7f095c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0961bef700 1 -- 172.21.15.92:0/1204727472 >> v1:172.21.15.92:6789/0 conn(0x7f095c0aed50 legacy=0x7f095c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0961bef700 1 -- 172.21.15.92:0/1204727472 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c071d90 con 0x7f095c0ce3d0 2023-06-09T04:35:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0961bef700 1 --2- 172.21.15.92:0/1204727472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c0713b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f095c070920 tx=0x7f094c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a002a26999e8099 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0960bed700 1 -- 172.21.15.92:0/1204727472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00b870 con 0x7f095c0ce3d0 2023-06-09T04:35:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0960bed700 1 -- 172.21.15.92:0/1204727472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00b9d0 con 0x7f095c0ce3d0 2023-06-09T04:35:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.105+0000 7f0960bed700 1 -- 172.21.15.92:0/1204727472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00bd60 con 0x7f095c0ce3d0 2023-06-09T04:35:57.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 -- 172.21.15.92:0/1204727472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 msgr2=0x7f095c0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 --2- 172.21.15.92:0/1204727472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c0713b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f095c070920 tx=0x7f094c009bd0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 -- 172.21.15.92:0/1204727472 shutdown_connections 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 --2- 172.21.15.92:0/1204727472 >> v2:172.21.15.123:3300/0 conn(0x7f095c15d470 0x7f095c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 --2- 172.21.15.92:0/1204727472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c0713b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 -- 172.21.15.92:0/1204727472 >> 172.21.15.92:0/1204727472 conn(0x7f095c0cc380 msgr2=0x7f095c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 -- 172.21.15.92:0/1204727472 shutdown_connections 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.106+0000 7f0963e53700 1 -- 172.21.15.92:0/1204727472 wait complete. 2023-06-09T04:35:57.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.107+0000 7f0963e53700 1 Processor -- start 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.107+0000 7f0963e53700 1 -- start start 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.107+0000 7f0963e53700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095c0aed50 0x7f095c15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.107+0000 7f0963e53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c1eff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0963e53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f095c15d470 0x7f095c0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0963e53700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c0c6f40 con 0x7f095c0ce3d0 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0963e53700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c15bb60 con 0x7f095c15d470 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0963e53700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c0c9030 con 0x7f095c0aed50 2023-06-09T04:35:57.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c1eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c1eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39026/0 (socket says 172.21.15.92:39026) 2023-06-09T04:35:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 -- 172.21.15.92:0/3945614599 learned_addr learned my addr 172.21.15.92:0/3945614599 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0961bef700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095c0aed50 0x7f095c15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09623f0700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f095c15d470 0x7f095c0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 -- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095c0aed50 msgr2=0x7f095c15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095c0aed50 0x7f095c15b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 -- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f095c15d470 msgr2=0x7f095c0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f095c15d470 0x7f095c0cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 -- 172.21.15.92:0/3945614599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c009550 con 0x7f095c0ce3d0 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0961bef700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095c0aed50 0x7f095c15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09613ee700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c1eff70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f095800c500 tx=0x7f095800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09527fc700 1 -- 172.21.15.92:0/3945614599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958005cb0 con 0x7f095c0ce3d0 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09527fc700 1 -- 172.21.15.92:0/3945614599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958005e10 con 0x7f095c0ce3d0 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c0c9310 con 0x7f095c0ce3d0 2023-06-09T04:35:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f09527fc700 1 -- 172.21.15.92:0/3945614599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09580072d0 con 0x7f095c0ce3d0 2023-06-09T04:35:57.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.108+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c15c970 con 0x7f095c0ce3d0 2023-06-09T04:35:57.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.109+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c06be50 con 0x7f095c0ce3d0 2023-06-09T04:35:57.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.110+0000 7f09527fc700 1 -- 172.21.15.92:0/3945614599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0958007430 con 0x7f095c0ce3d0 2023-06-09T04:35:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.110+0000 7f09527fc700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f094006ad10 0x7f094006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.110+0000 7f09527fc700 1 -- 172.21.15.92:0/3945614599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09580887d0 con 0x7f095c0ce3d0 2023-06-09T04:35:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.110+0000 7f0961bef700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f094006ad10 0x7f094006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.110+0000 7f0961bef700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f094006ad10 0x7f094006d1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.115+0000 7f09527fc700 1 -- 172.21.15.92:0/3945614599 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0958053930 con 0x7f095c0ce3d0 2023-06-09T04:35:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.302+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f095c1596c0 con 0x7f095c0ce3d0 2023-06-09T04:35:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.302+0000 7f09527fc700 1 -- 172.21.15.92:0/3945614599 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f095c1596c0 con 0x7f095c0ce3d0 2023-06-09T04:35:57.303 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:57.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f094006ad10 msgr2=0x7f094006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f094006ad10 0x7f094006d1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c008030 comp rx=0 tx=0).stop 2023-06-09T04:35:57.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 msgr2=0x7f095c1eff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c1eff70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f095800c500 tx=0x7f095800c8c0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 shutdown_connections 2023-06-09T04:35:57.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f094006ad10 0x7f094006d1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f095c15d470 0x7f095c0cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095c0aed50 0x7f095c15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 --2- 172.21.15.92:0/3945614599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095c0ce3d0 0x7f095c1eff70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 >> 172.21.15.92:0/3945614599 conn(0x7f095c0cc380 msgr2=0x7f095c0b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 shutdown_connections 2023-06-09T04:35:57.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.304+0000 7f0963e53700 1 -- 172.21.15.92:0/3945614599 wait complete. 2023-06-09T04:35:57.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-09T04:35:57.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.439+0000 7f2bb8008700 1 Processor -- start 2023-06-09T04:35:57.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.439+0000 7f2bb8008700 1 -- start start 2023-06-09T04:35:57.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb8008700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb0166f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb8008700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb00d17c0 con 0x7f2bb00ced00 2023-06-09T04:35:57.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb8008700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb00cf150 con 0x7f2bb00b2070 2023-06-09T04:35:57.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb8008700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00d0720 con 0x7f2bb00cfc40 2023-06-09T04:35:57.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb55a3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2bb00ced00 0x7f2bb00cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37366/0 (socket says 172.21.15.92:37366) 2023-06-09T04:35:57.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb55a3700 1 -- 172.21.15.92:0/3222505005 learned_addr learned my addr 172.21.15.92:0/3222505005 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:57.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb65a5700 1 --2- 172.21.15.92:0/3222505005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb0166f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb4da2700 1 -- 172.21.15.92:0/3222505005 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432132191 0 0) 0x7f2ba0001110 con 0x7f2bb00ced00 2023-06-09T04:35:57.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb4da2700 1 -- 172.21.15.92:0/3222505005 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b9c0035b0 con 0x7f2bb00ced00 2023-06-09T04:35:57.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb65a5700 1 -- 172.21.15.92:0/3222505005 >> v1:172.21.15.123:6789/0 conn(0x7f2bb00b2070 legacy=0x7f2bb00cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb4da2700 1 -- 172.21.15.92:0/3222505005 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92907128 0 0) 0x7f2bac001110 con 0x7f2bb00b2070 2023-06-09T04:35:57.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb65a5700 1 -- 172.21.15.92:0/3222505005 >> v1:172.21.15.92:6789/0 conn(0x7f2bb00ced00 legacy=0x7f2bb00cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.440+0000 7f2bb65a5700 1 -- 172.21.15.92:0/3222505005 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb00d0a00 con 0x7f2bb00cfc40 2023-06-09T04:35:57.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb65a5700 1 --2- 172.21.15.92:0/3222505005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb0166f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009dc0 tx=0x7f2ba800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e2d0834a4389376 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb4da2700 1 -- 172.21.15.92:0/3222505005 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800e070 con 0x7f2bb00cfc40 2023-06-09T04:35:57.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb4da2700 1 -- 172.21.15.92:0/3222505005 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba800bb20 con 0x7f2bb00cfc40 2023-06-09T04:35:57.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb4da2700 1 -- 172.21.15.92:0/3222505005 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800bed0 con 0x7f2bb00cfc40 2023-06-09T04:35:57.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb8008700 1 -- 172.21.15.92:0/3222505005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 msgr2=0x7f2bb0166f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb8008700 1 --2- 172.21.15.92:0/3222505005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb0166f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009dc0 tx=0x7f2ba800b330 comp rx=0 tx=0).stop 2023-06-09T04:35:57.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb8008700 1 -- 172.21.15.92:0/3222505005 shutdown_connections 2023-06-09T04:35:57.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb8008700 1 --2- 172.21.15.92:0/3222505005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb0166f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.441+0000 7f2bb8008700 1 -- 172.21.15.92:0/3222505005 >> 172.21.15.92:0/3222505005 conn(0x7f2bb0065a50 msgr2=0x7f2bb0065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:57.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.442+0000 7f2bb8008700 1 -- 172.21.15.92:0/3222505005 shutdown_connections 2023-06-09T04:35:57.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.442+0000 7f2bb8008700 1 -- 172.21.15.92:0/3222505005 wait complete. 2023-06-09T04:35:57.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.443+0000 7f2bb8008700 1 Processor -- start 2023-06-09T04:35:57.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.443+0000 7f2bb8008700 1 -- start start 2023-06-09T04:35:57.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb8008700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb00b2070 0x7f2bb01510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb8008700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb00ced00 0x7f2bb00c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb8008700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb00c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb8008700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb01580f0 con 0x7f2bb00b2070 2023-06-09T04:35:57.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb8008700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb0157200 con 0x7f2bb00ced00 2023-06-09T04:35:57.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb8008700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb01574e0 con 0x7f2bb00cfc40 2023-06-09T04:35:57.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb65a5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb00c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb5da4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb00b2070 0x7f2bb01510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb65a5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb00c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:47950/0 (socket says 172.21.15.92:47950) 2023-06-09T04:35:57.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb65a5700 1 -- 172.21.15.92:0/1048931970 learned_addr learned my addr 172.21.15.92:0/1048931970 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:57.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb55a3700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb00ced00 0x7f2bb00c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb5da4700 1 -- 172.21.15.92:0/1048931970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 msgr2=0x7f2bb00c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb5da4700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb00c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb5da4700 1 -- 172.21.15.92:0/1048931970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb00ced00 msgr2=0x7f2bb00c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb5da4700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb00ced00 0x7f2bb00c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.444+0000 7f2bb5da4700 1 -- 172.21.15.92:0/1048931970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba8009a70 con 0x7f2bb00b2070 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.445+0000 7f2bb5da4700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb00b2070 0x7f2bb01510f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c700 tx=0x7f2bac00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.445+0000 7f2ba67fc700 1 -- 172.21.15.92:0/1048931970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0052d0 con 0x7f2bb00b2070 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.445+0000 7f2ba67fc700 1 -- 172.21.15.92:0/1048931970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac005430 con 0x7f2bb00b2070 2023-06-09T04:35:57.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.445+0000 7f2ba67fc700 1 -- 172.21.15.92:0/1048931970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0038c0 con 0x7f2bb00b2070 2023-06-09T04:35:57.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.445+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb01577c0 con 0x7f2bb00b2070 2023-06-09T04:35:57.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.445+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb0157d80 con 0x7f2bb00b2070 2023-06-09T04:35:57.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.446+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb00740e0 con 0x7f2bb00b2070 2023-06-09T04:35:57.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.446+0000 7f2ba67fc700 1 -- 172.21.15.92:0/1048931970 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bac003a20 con 0x7f2bb00b2070 2023-06-09T04:35:57.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.447+0000 7f2ba67fc700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2b9406ace0 0x7f2b9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.447+0000 7f2bb55a3700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2b9406ace0 0x7f2b9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.447+0000 7f2ba67fc700 1 -- 172.21.15.92:0/1048931970 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2bac0885e0 con 0x7f2bb00b2070 2023-06-09T04:35:57.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.447+0000 7f2bb55a3700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2b9406ace0 0x7f2b9406d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2bb015f690 tx=0x7f2ba0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.451+0000 7f2ba67fc700 1 -- 172.21.15.92:0/1048931970 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2bac053690 con 0x7f2bb00b2070 2023-06-09T04:35:57.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.652+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2bb00cf150 con 0x7f2bb00b2070 2023-06-09T04:35:57.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.652+0000 7f2ba67fc700 1 -- 172.21.15.92:0/1048931970 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2bb00cf150 con 0x7f2bb00b2070 2023-06-09T04:35:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2b9406ace0 msgr2=0x7f2b9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2b9406ace0 0x7f2b9406d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2bb015f690 tx=0x7f2ba0009040 comp rx=0 tx=0).stop 2023-06-09T04:35:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb00b2070 msgr2=0x7f2bb01510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb00b2070 0x7f2bb01510f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c700 tx=0x7f2bac00ca10 comp rx=0 tx=0).stop 2023-06-09T04:35:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 shutdown_connections 2023-06-09T04:35:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f2b9406ace0 0x7f2b9406d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb00cfc40 0x7f2bb00c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bb00b2070 0x7f2bb01510f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 --2- 172.21.15.92:0/1048931970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb00ced00 0x7f2bb00c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 >> 172.21.15.92:0/1048931970 conn(0x7f2bb0065a50 msgr2=0x7f2bb00b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:57.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 shutdown_connections 2023-06-09T04:35:57.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.654+0000 7f2bb8008700 1 -- 172.21.15.92:0/1048931970 wait complete. 2023-06-09T04:35:57.657 INFO:teuthology.orchestra.run.smithi092.stderr:listed 0 entries 2023-06-09T04:35:57.666 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-06-09T04:35:57.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:57.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-06-09T04:35:57.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:35:57.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:35:57.717 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-09T04:35:57.735 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-06-09T04:35:57.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:35:57.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.855+0000 7f96c9e55700 1 Processor -- start 2023-06-09T04:35:57.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.855+0000 7f96c9e55700 1 -- start start 2023-06-09T04:35:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c9e55700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96c406ee50 0x7f96c40c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c9e55700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96c40c9740 0x7f96c406cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c9e55700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c4158e60 con 0x7f96c4070710 2023-06-09T04:35:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c9e55700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c4159ae0 con 0x7f96c40c9740 2023-06-09T04:35:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c9e55700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c4159140 con 0x7f96c406ee50 2023-06-09T04:35:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96c40c9740 0x7f96c406cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96c4070710 0x7f96c4070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37378/0 (socket says 172.21.15.92:37378) 2023-06-09T04:35:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f96c40c9740 0x7f96c406cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39050/0 (socket says 172.21.15.92:39050) 2023-06-09T04:35:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c37fe700 1 -- 172.21.15.92:0/2606741147 learned_addr learned my addr 172.21.15.92:0/2606741147 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c2ffd700 1 --2- 172.21.15.92:0/2606741147 >> v2:172.21.15.123:3300/0 conn(0x7f96c406ee50 0x7f96c40c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c3fff700 1 -- 172.21.15.92:0/2606741147 >> v2:172.21.15.123:3300/0 conn(0x7f96c406ee50 msgr2=0x7f96c40c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c3fff700 1 --2- 172.21.15.92:0/2606741147 >> v2:172.21.15.123:3300/0 conn(0x7f96c406ee50 0x7f96c40c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c3fff700 1 -- 172.21.15.92:0/2606741147 >> v1:172.21.15.92:6789/0 conn(0x7f96c4070710 legacy=0x7f96c4070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c3fff700 1 -- 172.21.15.92:0/2606741147 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4159420 con 0x7f96c40c9740 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c27fc700 1 -- 172.21.15.92:0/2606741147 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416450182 0 0) 0x7f96c4158e60 con 0x7f96c4070710 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c2ffd700 1 --2- 172.21.15.92:0/2606741147 >> v2:172.21.15.123:3300/0 conn(0x7f96c406ee50 0x7f96c40c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c3fff700 1 --2- 172.21.15.92:0/2606741147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c406cf00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f96c40701c0 tx=0x7f96bc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6625e1cae9b25b8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c27fc700 1 -- 172.21.15.92:0/2606741147 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00f070 con 0x7f96c40c9740 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c27fc700 1 -- 172.21.15.92:0/2606741147 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc00a2e0 con 0x7f96c40c9740 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.856+0000 7f96c27fc700 1 -- 172.21.15.92:0/2606741147 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc00cc00 con 0x7f96c40c9740 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 -- 172.21.15.92:0/2606741147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 msgr2=0x7f96c406cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 --2- 172.21.15.92:0/2606741147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c406cf00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f96c40701c0 tx=0x7f96bc00c330 comp rx=0 tx=0).stop 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 -- 172.21.15.92:0/2606741147 shutdown_connections 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 --2- 172.21.15.92:0/2606741147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c406cf00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 --2- 172.21.15.92:0/2606741147 >> v2:172.21.15.123:3300/0 conn(0x7f96c406ee50 0x7f96c40c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 -- 172.21.15.92:0/2606741147 >> 172.21.15.92:0/2606741147 conn(0x7f96c40b0880 msgr2=0x7f96c40b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:57.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 -- 172.21.15.92:0/2606741147 shutdown_connections 2023-06-09T04:35:57.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.857+0000 7f96c9e55700 1 -- 172.21.15.92:0/2606741147 wait complete. 2023-06-09T04:35:57.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.858+0000 7f96c9e55700 1 Processor -- start 2023-06-09T04:35:57.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.858+0000 7f96c9e55700 1 -- start start 2023-06-09T04:35:57.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c9e55700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96c4070710 0x7f96c414a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c9e55700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c40b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c9e55700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96c40b2ad0 0x7f96c40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c9e55700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c415b410 con 0x7f96c40c9740 2023-06-09T04:35:57.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c9e55700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40b32c0 con 0x7f96c40b2ad0 2023-06-09T04:35:57.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c9e55700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c40b2fe0 con 0x7f96c4070710 2023-06-09T04:35:57.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c40b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c40b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39054/0 (socket says 172.21.15.92:39054) 2023-06-09T04:35:57.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c2ffd700 1 -- 172.21.15.92:0/1507388031 learned_addr learned my addr 172.21.15.92:0/1507388031 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:57.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.859+0000 7f96c3fff700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96c40b2ad0 0x7f96c40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c37fe700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96c4070710 0x7f96c414a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c2ffd700 1 -- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96c4070710 msgr2=0x7f96c414a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c2ffd700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96c4070710 0x7f96c414a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c2ffd700 1 -- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96c40b2ad0 msgr2=0x7f96c40b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c2ffd700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96c40b2ad0 0x7f96c40b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c2ffd700 1 -- 172.21.15.92:0/1507388031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc00ab80 con 0x7f96c40c9740 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c2ffd700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c40b2590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f96b800e810 tx=0x7f96b800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96abfff700 1 -- 172.21.15.92:0/1507388031 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800c070 con 0x7f96c40c9740 2023-06-09T04:35:57.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96abfff700 1 -- 172.21.15.92:0/1507388031 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b8004230 con 0x7f96c40c9740 2023-06-09T04:35:57.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c415b640 con 0x7f96c40c9740 2023-06-09T04:35:57.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96abfff700 1 -- 172.21.15.92:0/1507388031 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80036a0 con 0x7f96c40c9740 2023-06-09T04:35:57.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.860+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c415bc00 con 0x7f96c40c9740 2023-06-09T04:35:57.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.861+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c40740e0 con 0x7f96c40c9740 2023-06-09T04:35:57.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.862+0000 7f96abfff700 1 -- 172.21.15.92:0/1507388031 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96b8003800 con 0x7f96c40c9740 2023-06-09T04:35:57.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.862+0000 7f96abfff700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96b006ace0 0x7f96b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:57.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.862+0000 7f96abfff700 1 -- 172.21.15.92:0/1507388031 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96b80890a0 con 0x7f96c40c9740 2023-06-09T04:35:57.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.862+0000 7f96c37fe700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96b006ace0 0x7f96b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:57.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.862+0000 7f96c37fe700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96b006ace0 0x7f96b006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96b4007fd0 tx=0x7f96b4007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:57.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:57.867+0000 7f96abfff700 1 -- 172.21.15.92:0/1507388031 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f96b80541e0 con 0x7f96c40c9740 2023-06-09T04:35:58.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.099+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96c40c8b40 con 0x7f96c40c9740 2023-06-09T04:35:58.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.099+0000 7f96abfff700 1 -- 172.21.15.92:0/1507388031 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f96b80094c0 con 0x7f96c40c9740 2023-06-09T04:35:58.100 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:58.101 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:35:58.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96b006ace0 msgr2=0x7f96b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96b006ace0 0x7f96b006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96b4007fd0 tx=0x7f96b4007de0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 msgr2=0x7f96c40b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c40b2590 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f96b800e810 tx=0x7f96b800eb20 comp rx=0 tx=0).stop 2023-06-09T04:35:58.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 shutdown_connections 2023-06-09T04:35:58.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f96b006ace0 0x7f96b006d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96c40b2ad0 0x7f96c40b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96c4070710 0x7f96c414a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 --2- 172.21.15.92:0/1507388031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96c40c9740 0x7f96c40b2590 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.101+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 >> 172.21.15.92:0/1507388031 conn(0x7f96c40b0880 msgr2=0x7f96c40b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:58.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.102+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 shutdown_connections 2023-06-09T04:35:58.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.102+0000 7f96c9e55700 1 -- 172.21.15.92:0/1507388031 wait complete. 2023-06-09T04:35:58.105 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:35:58.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:35:58.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.236+0000 7fa9401a3700 1 Processor -- start 2023-06-09T04:35:58.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.236+0000 7fa9401a3700 1 -- start start 2023-06-09T04:35:58.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa9401a3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa9401a3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9380cbdd0 0x7fa9381608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa9401a3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93806b720 con 0x7fa9380cb960 2023-06-09T04:35:58.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa9401a3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93815ca60 con 0x7fa9380cbdd0 2023-06-09T04:35:58.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa9401a3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380cd050 con 0x7fa93815c690 2023-06-09T04:35:58.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa93e740700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9380cbdd0 0x7fa9381608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa93e740700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9380cbdd0 0x7fa9381608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39066/0 (socket says 172.21.15.92:39066) 2023-06-09T04:35:58.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.237+0000 7fa93e740700 1 -- 172.21.15.92:0/74506629 learned_addr learned my addr 172.21.15.92:0/74506629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:58.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93df3f700 1 --2- 172.21.15.92:0/74506629 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93e740700 1 -- 172.21.15.92:0/74506629 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 msgr2=0x7fa9380ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:58.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93e740700 1 --2- 172.21.15.92:0/74506629 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93e740700 1 -- 172.21.15.92:0/74506629 >> v1:172.21.15.123:6789/0 conn(0x7fa9380cb960 legacy=0x7fa9381601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93e740700 1 -- 172.21.15.92:0/74506629 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380cd330 con 0x7fa9380cbdd0 2023-06-09T04:35:58.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93e740700 1 --2- 172.21.15.92:0/74506629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381608f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa930009e50 tx=0x7fa93000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2d323539c3ffb3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93cf3d700 1 -- 172.21.15.92:0/74506629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000e070 con 0x7fa9380cbdd0 2023-06-09T04:35:58.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93cf3d700 1 -- 172.21.15.92:0/74506629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93000b9c0 con 0x7fa9380cbdd0 2023-06-09T04:35:58.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.238+0000 7fa93cf3d700 1 -- 172.21.15.92:0/74506629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000bd70 con 0x7fa9380cbdd0 2023-06-09T04:35:58.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 -- 172.21.15.92:0/74506629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 msgr2=0x7fa9381608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 --2- 172.21.15.92:0/74506629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381608f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa930009e50 tx=0x7fa93000b330 comp rx=0 tx=0).stop 2023-06-09T04:35:58.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 -- 172.21.15.92:0/74506629 shutdown_connections 2023-06-09T04:35:58.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 --2- 172.21.15.92:0/74506629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381608f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 --2- 172.21.15.92:0/74506629 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 -- 172.21.15.92:0/74506629 >> 172.21.15.92:0/74506629 conn(0x7fa9380b1800 msgr2=0x7fa93806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:58.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 -- 172.21.15.92:0/74506629 shutdown_connections 2023-06-09T04:35:58.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.239+0000 7fa9401a3700 1 -- 172.21.15.92:0/74506629 wait complete. 2023-06-09T04:35:58.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.240+0000 7fa9401a3700 1 Processor -- start 2023-06-09T04:35:58.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.240+0000 7fa9401a3700 1 -- start start 2023-06-09T04:35:58.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa9401a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9380cb960 0x7fa938146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa9401a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa9401a3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa9401a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381e6350 con 0x7fa9380cbdd0 2023-06-09T04:35:58.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa9401a3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381e6630 con 0x7fa9380cb960 2023-06-09T04:35:58.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa9401a3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381e6910 con 0x7fa93815c690 2023-06-09T04:35:58.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39072/0 (socket says 172.21.15.92:39072) 2023-06-09T04:35:58.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 -- 172.21.15.92:0/1263454006 learned_addr learned my addr 172.21.15.92:0/1263454006 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:58.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93df3f700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9380cb960 0x7fa938146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93e740700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 -- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 msgr2=0x7fa9380b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 -- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9380cb960 msgr2=0x7fa938146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9380cb960 0x7fa938146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 -- 172.21.15.92:0/1263454006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa930009b30 con 0x7fa9380cbdd0 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93e740700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93df3f700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9380cb960 0x7fa938146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa93d73e700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381e73f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa92800c4e0 tx=0x7fa92800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa92e7fc700 1 -- 172.21.15.92:0/1263454006 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280052d0 con 0x7fa9380cbdd0 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa92e7fc700 1 -- 172.21.15.92:0/1263454006 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa928005430 con 0x7fa9380cbdd0 2023-06-09T04:35:58.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9381e6bf0 con 0x7fa9380cbdd0 2023-06-09T04:35:58.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.241+0000 7fa92e7fc700 1 -- 172.21.15.92:0/1263454006 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280038c0 con 0x7fa9380cbdd0 2023-06-09T04:35:58.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.242+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa938147320 con 0x7fa9380cbdd0 2023-06-09T04:35:58.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.243+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93806da80 con 0x7fa9380cbdd0 2023-06-09T04:35:58.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.243+0000 7fa92e7fc700 1 -- 172.21.15.92:0/1263454006 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa928003a20 con 0x7fa9380cbdd0 2023-06-09T04:35:58.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.243+0000 7fa92e7fc700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa91c06ace0 0x7fa91c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.243+0000 7fa92e7fc700 1 -- 172.21.15.92:0/1263454006 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa9280889a0 con 0x7fa9380cbdd0 2023-06-09T04:35:58.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.243+0000 7fa93df3f700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa91c06ace0 0x7fa91c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.244+0000 7fa93df3f700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa91c06ace0 0x7fa91c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa934009820 tx=0x7fa934008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.247+0000 7fa92e7fc700 1 -- 172.21.15.92:0/1263454006 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa928053a10 con 0x7fa9380cbdd0 2023-06-09T04:35:58.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.435+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9381e7110 con 0x7fa9380cbdd0 2023-06-09T04:35:58.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.436+0000 7fa92e7fc700 1 -- 172.21.15.92:0/1263454006 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fa928057030 con 0x7fa9380cbdd0 2023-06-09T04:35:58.437 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:35:58.438 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":25,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:58.224650+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:35:58.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa91c06ace0 msgr2=0x7fa91c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa91c06ace0 0x7fa91c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa934009820 tx=0x7fa934008040 comp rx=0 tx=0).stop 2023-06-09T04:35:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 msgr2=0x7fa9381e73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381e73f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa92800c4e0 tx=0x7fa92800c8a0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 shutdown_connections 2023-06-09T04:35:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa91c06ace0 0x7fa91c06d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa93815c690 0x7fa9380b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9380cb960 0x7fa938146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 --2- 172.21.15.92:0/1263454006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9380cbdd0 0x7fa9381e73f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 >> 172.21.15.92:0/1263454006 conn(0x7fa9380b1800 msgr2=0x7fa93815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:58.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 shutdown_connections 2023-06-09T04:35:58.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.438+0000 7fa9401a3700 1 -- 172.21.15.92:0/1263454006 wait complete. 2023-06-09T04:35:58.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:35:15.100764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:35:16.382513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:35:58.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:35:58.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.575+0000 7fa59030d700 1 Processor -- start 2023-06-09T04:35:58.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.575+0000 7fa59030d700 1 -- start start 2023-06-09T04:35:58.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa59030d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa5880cf440 0x7fa58815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa59030d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5880cce60 0x7fa58806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa59030d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58815d040 con 0x7fa588069db0 2023-06-09T04:35:58.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa59030d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa588160ab0 con 0x7fa5880cf440 2023-06-09T04:35:58.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa59030d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58815ca10 con 0x7fa5880cce60 2023-06-09T04:35:58.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58d8a8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa5880cf440 0x7fa58815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58d8a8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa5880cf440 0x7fa58815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39080/0 (socket says 172.21.15.92:39080) 2023-06-09T04:35:58.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58d8a8700 1 -- 172.21.15.92:0/2601145883 learned_addr learned my addr 172.21.15.92:0/2601145883 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:58.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58e8aa700 1 --2- 172.21.15.92:0/2601145883 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5880cce60 0x7fa58806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58d8a8700 1 -- 172.21.15.92:0/2601145883 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5880cce60 msgr2=0x7fa58806ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:58.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58d8a8700 1 --2- 172.21.15.92:0/2601145883 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5880cce60 0x7fa58806ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58d8a8700 1 -- 172.21.15.92:0/2601145883 >> v1:172.21.15.123:6789/0 conn(0x7fa588069db0 legacy=0x7fa58806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:58.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.576+0000 7fa58d8a8700 1 -- 172.21.15.92:0/2601145883 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588160d50 con 0x7fa5880cf440 2023-06-09T04:35:58.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa58d8a8700 1 --2- 172.21.15.92:0/2601145883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cf440 0x7fa58815d4d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098f0 tx=0x7fa584009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4adee89fb5afc89 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa58d0a7700 1 -- 172.21.15.92:0/2601145883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584004230 con 0x7fa5880cf440 2023-06-09T04:35:58.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa58d0a7700 1 -- 172.21.15.92:0/2601145883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584004390 con 0x7fa5880cf440 2023-06-09T04:35:58.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa58d0a7700 1 -- 172.21.15.92:0/2601145883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840036a0 con 0x7fa5880cf440 2023-06-09T04:35:58.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 -- 172.21.15.92:0/2601145883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cf440 msgr2=0x7fa58815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 --2- 172.21.15.92:0/2601145883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cf440 0x7fa58815d4d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa5840098f0 tx=0x7fa584009c00 comp rx=0 tx=0).stop 2023-06-09T04:35:58.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 -- 172.21.15.92:0/2601145883 shutdown_connections 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 --2- 172.21.15.92:0/2601145883 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa5880cce60 0x7fa58806ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 --2- 172.21.15.92:0/2601145883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cf440 0x7fa58815d4d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 -- 172.21.15.92:0/2601145883 >> 172.21.15.92:0/2601145883 conn(0x7fa5880cc380 msgr2=0x7fa5880cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 -- 172.21.15.92:0/2601145883 shutdown_connections 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.577+0000 7fa59030d700 1 -- 172.21.15.92:0/2601145883 wait complete. 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 Processor -- start 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 -- start start 2023-06-09T04:35:58.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa588069db0 0x7fa5881f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cce60 0x7fa5880b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5880cf440 0x7fa5880b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880c5f80 con 0x7fa5880cce60 2023-06-09T04:35:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5881f5cd0 con 0x7fa5880cf440 2023-06-09T04:35:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa59030d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880b5130 con 0x7fa588069db0 2023-06-09T04:35:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa58d8a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cce60 0x7fa5880b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa58d8a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cce60 0x7fa5880b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39086/0 (socket says 172.21.15.92:39086) 2023-06-09T04:35:58.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.579+0000 7fa58d8a8700 1 -- 172.21.15.92:0/3626757702 learned_addr learned my addr 172.21.15.92:0/3626757702 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:58.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58e0a9700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa588069db0 0x7fa5881f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58e8aa700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5880cf440 0x7fa5880b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58d8a8700 1 -- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa588069db0 msgr2=0x7fa5881f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58d8a8700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa588069db0 0x7fa5881f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58d8a8700 1 -- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5880cf440 msgr2=0x7fa5880b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58d8a8700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5880cf440 0x7fa5880b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58d8a8700 1 -- 172.21.15.92:0/3626757702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009550 con 0x7fa5880cce60 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58e8aa700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5880cf440 0x7fa5880b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58e0a9700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa588069db0 0x7fa5881f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa58d8a8700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cce60 0x7fa5880b3520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa584009840 tx=0x7fa584004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa57effd700 1 -- 172.21.15.92:0/3626757702 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584003e50 con 0x7fa5880cce60 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa57effd700 1 -- 172.21.15.92:0/3626757702 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58400b660 con 0x7fa5880cce60 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa588156360 con 0x7fa5880cce60 2023-06-09T04:35:58.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa57effd700 1 -- 172.21.15.92:0/3626757702 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400ba10 con 0x7fa5880cce60 2023-06-09T04:35:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.580+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5880b4370 con 0x7fa5880cce60 2023-06-09T04:35:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.581+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58806be50 con 0x7fa5880cce60 2023-06-09T04:35:58.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.581+0000 7fa57effd700 1 -- 172.21.15.92:0/3626757702 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa58400bb70 con 0x7fa5880cce60 2023-06-09T04:35:58.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.582+0000 7fa57effd700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa56c06ace0 0x7fa56c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.582+0000 7fa58e0a9700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa56c06ace0 0x7fa56c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.582+0000 7fa57effd700 1 -- 172.21.15.92:0/3626757702 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa58408ab80 con 0x7fa5880cce60 2023-06-09T04:35:58.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.582+0000 7fa58e0a9700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa56c06ace0 0x7fa56c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa578003fc0 tx=0x7fa578009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.586+0000 7fa57effd700 1 -- 172.21.15.92:0/3626757702 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa584055c80 con 0x7fa5880cce60 2023-06-09T04:35:58.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.774+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa58804e8e0 con 0x7fa5880cce60 2023-06-09T04:35:58.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.774+0000 7fa57effd700 1 -- 172.21.15.92:0/3626757702 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5840592a0 con 0x7fa5880cce60 2023-06-09T04:35:58.775 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:35:58.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa56c06ace0 msgr2=0x7fa56c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa56c06ace0 0x7fa56c06d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa578003fc0 tx=0x7fa578009040 comp rx=0 tx=0).stop 2023-06-09T04:35:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cce60 msgr2=0x7fa5880b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cce60 0x7fa5880b3520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa584009840 tx=0x7fa584004750 comp rx=0 tx=0).stop 2023-06-09T04:35:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 shutdown_connections 2023-06-09T04:35:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa56c06ace0 0x7fa56c06d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5880cf440 0x7fa5880b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa588069db0 0x7fa5881f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 --2- 172.21.15.92:0/3626757702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5880cce60 0x7fa5880b3520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 >> 172.21.15.92:0/3626757702 conn(0x7fa5880cc380 msgr2=0x7fa58815e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:58.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 shutdown_connections 2023-06-09T04:35:58.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.776+0000 7fa59030d700 1 -- 172.21.15.92:0/3626757702 wait complete. 2023-06-09T04:35:58.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:35:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.910+0000 7f1fad1c8700 1 Processor -- start 2023-06-09T04:35:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.910+0000 7f1fad1c8700 1 -- start start 2023-06-09T04:35:58.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.910+0000 7f1fad1c8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1fa80aed50 0x7f1fa80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.910+0000 7f1fad1c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa815d470 0x7f1fa815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.910+0000 7f1fad1c8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa80ce190 con 0x7f1fa80ce3d0 2023-06-09T04:35:58.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.910+0000 7f1fad1c8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa8072350 con 0x7f1fa80aed50 2023-06-09T04:35:58.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.910+0000 7f1fad1c8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8071ab0 con 0x7f1fa815d470 2023-06-09T04:35:58.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1fa80aed50 0x7f1fa80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa6d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1fa80ce3d0 0x7f1fa80ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37424/0 (socket says 172.21.15.92:37424) 2023-06-09T04:35:58.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa6d9d700 1 -- 172.21.15.92:0/2816164080 learned_addr learned my addr 172.21.15.92:0/2816164080 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:58.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa759e700 1 --2- 172.21.15.92:0/2816164080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa815d470 0x7f1fa815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa659c700 1 -- 172.21.15.92:0/2816164080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa815d470 msgr2=0x7f1fa815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa659c700 1 --2- 172.21.15.92:0/2816164080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa815d470 0x7f1fa815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa659c700 1 -- 172.21.15.92:0/2816164080 >> v1:172.21.15.92:6789/0 conn(0x7f1fa80ce3d0 legacy=0x7f1fa80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa5d9b700 1 -- 172.21.15.92:0/2816164080 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384865493 0 0) 0x7f1fa80ce190 con 0x7f1fa80ce3d0 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa659c700 1 -- 172.21.15.92:0/2816164080 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8071d90 con 0x7f1fa80aed50 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa659c700 1 --2- 172.21.15.92:0/2816164080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa80713b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009840 tx=0x7f1f9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c59d5274a0c49a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa5d9b700 1 -- 172.21.15.92:0/2816164080 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011830 con 0x7f1fa80aed50 2023-06-09T04:35:58.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa5d9b700 1 -- 172.21.15.92:0/2816164080 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c011990 con 0x7f1fa80aed50 2023-06-09T04:35:58.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.911+0000 7f1fa5d9b700 1 -- 172.21.15.92:0/2816164080 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011d20 con 0x7f1fa80aed50 2023-06-09T04:35:58.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 -- 172.21.15.92:0/2816164080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 msgr2=0x7f1fa80713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/2816164080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa80713b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009840 tx=0x7f1f9c009b50 comp rx=0 tx=0).stop 2023-06-09T04:35:58.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 -- 172.21.15.92:0/2816164080 shutdown_connections 2023-06-09T04:35:58.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/2816164080 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa815d470 0x7f1fa815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/2816164080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa80713b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 -- 172.21.15.92:0/2816164080 >> 172.21.15.92:0/2816164080 conn(0x7f1fa80cc380 msgr2=0x7f1fa80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:58.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 -- 172.21.15.92:0/2816164080 shutdown_connections 2023-06-09T04:35:58.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.912+0000 7f1fad1c8700 1 -- 172.21.15.92:0/2816164080 wait complete. 2023-06-09T04:35:58.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.913+0000 7f1fad1c8700 1 Processor -- start 2023-06-09T04:35:58.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.913+0000 7f1fad1c8700 1 -- start start 2023-06-09T04:35:58.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fad1c8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa81531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fad1c8700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa80ce3d0 0x7f1fa8157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fad1c8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa815d470 0x7f1fa8158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fad1c8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa815b6a0 con 0x7f1fa80aed50 2023-06-09T04:35:58.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fad1c8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8153730 con 0x7f1fa815d470 2023-06-09T04:35:58.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fad1c8700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa815aa20 con 0x7f1fa80ce3d0 2023-06-09T04:35:58.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa81531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa81531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39098/0 (socket says 172.21.15.92:39098) 2023-06-09T04:35:58.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 -- 172.21.15.92:0/3149126088 learned_addr learned my addr 172.21.15.92:0/3149126088 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:58.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa659c700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa80ce3d0 0x7f1fa8157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa759e700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa815d470 0x7f1fa8158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 -- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa80ce3d0 msgr2=0x7f1fa8157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa80ce3d0 0x7f1fa8157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 -- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa815d470 msgr2=0x7f1fa8158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:58.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa815d470 0x7f1fa8158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:58.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa6d9d700 1 -- 172.21.15.92:0/3149126088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0094a0 con 0x7f1fa80aed50 2023-06-09T04:35:58.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa659c700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa80ce3d0 0x7f1fa8157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:58.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.914+0000 7f1fa759e700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa815d470 0x7f1fa8158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:58.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.915+0000 7f1fa6d9d700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa81531f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f980011e0 tx=0x7f1f9800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.915+0000 7f1f8f7fe700 1 -- 172.21.15.92:0/3149126088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98004c80 con 0x7f1fa80aed50 2023-06-09T04:35:58.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.915+0000 7f1f8f7fe700 1 -- 172.21.15.92:0/3149126088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f98004e00 con 0x7f1fa80aed50 2023-06-09T04:35:58.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.915+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa815ad00 con 0x7f1fa80aed50 2023-06-09T04:35:58.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.915+0000 7f1f8f7fe700 1 -- 172.21.15.92:0/3149126088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98007310 con 0x7f1fa80aed50 2023-06-09T04:35:58.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.915+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa815b2c0 con 0x7f1fa80aed50 2023-06-09T04:35:58.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.916+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa81596c0 con 0x7f1fa80aed50 2023-06-09T04:35:58.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.916+0000 7f1f8f7fe700 1 -- 172.21.15.92:0/3149126088 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f98007470 con 0x7f1fa80aed50 2023-06-09T04:35:58.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.916+0000 7f1f8f7fe700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f9406ace0 0x7f1f9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:58.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.917+0000 7f1f8f7fe700 1 -- 172.21.15.92:0/3149126088 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1f980891d0 con 0x7f1fa80aed50 2023-06-09T04:35:58.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.917+0000 7f1fa659c700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f9406ace0 0x7f1f9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:58.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.917+0000 7f1fa659c700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f9406ace0 0x7f1f9406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0090b0 tx=0x7f1f9c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:58.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:58.921+0000 7f1f8f7fe700 1 -- 172.21.15.92:0/3149126088 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fa81596c0 con 0x7f1fa80aed50 2023-06-09T04:35:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.107+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1fa81596c0 con 0x7f1fa80aed50 2023-06-09T04:35:59.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.107+0000 7f1f8f7fe700 1 -- 172.21.15.92:0/3149126088 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1fa81596c0 con 0x7f1fa80aed50 2023-06-09T04:35:59.108 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:35:59.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f9406ace0 msgr2=0x7f1f9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f9406ace0 0x7f1f9406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0090b0 tx=0x7f1f9c01c040 comp rx=0 tx=0).stop 2023-06-09T04:35:59.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 msgr2=0x7f1fa81531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa81531f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f980011e0 tx=0x7f1f9800cae0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 shutdown_connections 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f1f9406ace0 0x7f1f9406d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fa815d470 0x7f1fa8158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fa80aed50 0x7f1fa81531f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 --2- 172.21.15.92:0/3149126088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fa80ce3d0 0x7f1fa8157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.109+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 >> 172.21.15.92:0/3149126088 conn(0x7f1fa80cc380 msgr2=0x7f1fa8069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.110+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 shutdown_connections 2023-06-09T04:35:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.110+0000 7f1fad1c8700 1 -- 172.21.15.92:0/3149126088 wait complete. 2023-06-09T04:35:59.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:35:59.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.245+0000 7f0b964f2700 1 Processor -- start 2023-06-09T04:35:59.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.245+0000 7f0b964f2700 1 -- start start 2023-06-09T04:35:59.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.246+0000 7f0b964f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b900ce3d0 0x7f0b900713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.246+0000 7f0b964f2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b900aed50 0x7f0b900ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.246+0000 7f0b964f2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b900ce190 con 0x7f0b9015d470 2023-06-09T04:35:59.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.246+0000 7f0b964f2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b90072350 con 0x7f0b900ce3d0 2023-06-09T04:35:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.246+0000 7f0b964f2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b900cece0 con 0x7f0b900aed50 2023-06-09T04:35:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.246+0000 7f0b8ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b900ce3d0 0x7f0b900713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b900ce3d0 0x7f0b900713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39106/0 (socket says 172.21.15.92:39106) 2023-06-09T04:35:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8ffff700 1 -- 172.21.15.92:0/1688674100 learned_addr learned my addr 172.21.15.92:0/1688674100 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8f7fe700 1 --2- 172.21.15.92:0/1688674100 >> v2:172.21.15.123:3300/0 conn(0x7f0b900aed50 0x7f0b900ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8ffff700 1 -- 172.21.15.92:0/1688674100 >> v2:172.21.15.123:3300/0 conn(0x7f0b900aed50 msgr2=0x7f0b900ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8ffff700 1 --2- 172.21.15.92:0/1688674100 >> v2:172.21.15.123:3300/0 conn(0x7f0b900aed50 0x7f0b900ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8ffff700 1 -- 172.21.15.92:0/1688674100 >> v1:172.21.15.123:6789/0 conn(0x7f0b9015d470 legacy=0x7f0b901596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8ffff700 1 -- 172.21.15.92:0/1688674100 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b90071ab0 con 0x7f0b900ce3d0 2023-06-09T04:35:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8ffff700 1 --2- 172.21.15.92:0/1688674100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900713b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0b900c33a0 tx=0x7f0b8400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c75ad9571fa698 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8effd700 1 -- 172.21.15.92:0/1688674100 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400dd30 con 0x7f0b900ce3d0 2023-06-09T04:35:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8effd700 1 -- 172.21.15.92:0/1688674100 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8400de90 con 0x7f0b900ce3d0 2023-06-09T04:35:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.247+0000 7f0b8effd700 1 -- 172.21.15.92:0/1688674100 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84010690 con 0x7f0b900ce3d0 2023-06-09T04:35:59.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 -- 172.21.15.92:0/1688674100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 msgr2=0x7f0b900713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 --2- 172.21.15.92:0/1688674100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900713b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0b900c33a0 tx=0x7f0b8400be70 comp rx=0 tx=0).stop 2023-06-09T04:35:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 -- 172.21.15.92:0/1688674100 shutdown_connections 2023-06-09T04:35:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 --2- 172.21.15.92:0/1688674100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900713b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 --2- 172.21.15.92:0/1688674100 >> v2:172.21.15.123:3300/0 conn(0x7f0b900aed50 0x7f0b900ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 -- 172.21.15.92:0/1688674100 >> 172.21.15.92:0/1688674100 conn(0x7f0b900cc380 msgr2=0x7f0b900cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 -- 172.21.15.92:0/1688674100 shutdown_connections 2023-06-09T04:35:59.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.248+0000 7f0b964f2700 1 -- 172.21.15.92:0/1688674100 wait complete. 2023-06-09T04:35:59.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.249+0000 7f0b964f2700 1 Processor -- start 2023-06-09T04:35:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.249+0000 7f0b964f2700 1 -- start start 2023-06-09T04:35:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b964f2700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b900aed50 0x7f0b901587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b964f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b964f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9015d470 0x7f0b90151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b964f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900abf90 con 0x7f0b900ce3d0 2023-06-09T04:35:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b964f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900c7c80 con 0x7f0b9015d470 2023-06-09T04:35:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b964f2700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b90152300 con 0x7f0b900aed50 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39110/0 (socket says 172.21.15.92:39110) 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 -- 172.21.15.92:0/265580391 learned_addr learned my addr 172.21.15.92:0/265580391 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 -- 172.21.15.92:0/265580391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b900aed50 msgr2=0x7f0b901587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b900aed50 0x7f0b901587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 -- 172.21.15.92:0/265580391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9015d470 msgr2=0x7f0b90151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9015d470 0x7f0b90151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 -- 172.21.15.92:0/265580391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8400b7f0 con 0x7f0b900ce3d0 2023-06-09T04:35:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8f7fe700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900ac3b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d800 tx=0x7f0b8000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8cff9700 1 -- 172.21.15.92:0/265580391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800125f0 con 0x7f0b900ce3d0 2023-06-09T04:35:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8cff9700 1 -- 172.21.15.92:0/265580391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80012750 con 0x7f0b900ce3d0 2023-06-09T04:35:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b900c9510 con 0x7f0b900ce3d0 2023-06-09T04:35:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.250+0000 7f0b8cff9700 1 -- 172.21.15.92:0/265580391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80012b00 con 0x7f0b900ce3d0 2023-06-09T04:35:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.251+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b900ab8d0 con 0x7f0b900ce3d0 2023-06-09T04:35:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.251+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9006be50 con 0x7f0b900ce3d0 2023-06-09T04:35:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.252+0000 7f0b8cff9700 1 -- 172.21.15.92:0/265580391 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b80012c60 con 0x7f0b900ce3d0 2023-06-09T04:35:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.252+0000 7f0b8cff9700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0b7c06ace0 0x7f0b7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.252+0000 7f0b8cff9700 1 -- 172.21.15.92:0/265580391 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0b800898c0 con 0x7f0b900ce3d0 2023-06-09T04:35:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.252+0000 7f0b8ffff700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0b7c06ace0 0x7f0b7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.253+0000 7f0b8ffff700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0b7c06ace0 0x7f0b7c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400beb0 tx=0x7f0b8400bf90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.257+0000 7f0b8cff9700 1 -- 172.21.15.92:0/265580391 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0b80054930 con 0x7f0b900ce3d0 2023-06-09T04:35:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.443+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b900ac0d0 con 0x7f0b900ce3d0 2023-06-09T04:35:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.444+0000 7f0b8cff9700 1 -- 172.21.15.92:0/265580391 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b900ac0d0 con 0x7f0b900ce3d0 2023-06-09T04:35:59.444 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:35:59.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0b7c06ace0 msgr2=0x7f0b7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0b7c06ace0 0x7f0b7c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400beb0 tx=0x7f0b8400bf90 comp rx=0 tx=0).stop 2023-06-09T04:35:59.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 msgr2=0x7f0b900ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900ac3b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d800 tx=0x7f0b8000dbc0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 shutdown_connections 2023-06-09T04:35:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0b7c06ace0 0x7f0b7c06d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9015d470 0x7f0b90151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0b900aed50 0x7f0b901587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 --2- 172.21.15.92:0/265580391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b900ce3d0 0x7f0b900ac3b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 >> 172.21.15.92:0/265580391 conn(0x7f0b900cc380 msgr2=0x7f0b900cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:59.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 shutdown_connections 2023-06-09T04:35:59.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.446+0000 7f0b964f2700 1 -- 172.21.15.92:0/265580391 wait complete. 2023-06-09T04:35:59.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:35:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.581+0000 7fa1012c9700 1 Processor -- start 2023-06-09T04:35:59.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.581+0000 7fa1012c9700 1 -- start start 2023-06-09T04:35:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa1012c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 0x7fa0fc0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa1012c9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0fc06aec0 0x7fa0fc06b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa1012c9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc0ce720 con 0x7fa0fc15c5e0 2023-06-09T04:35:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa1012c9700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc0cea00 con 0x7fa0fc06aec0 2023-06-09T04:35:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa1012c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0cece0 con 0x7fa0fc0d0370 2023-06-09T04:35:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0faffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa0fc15c5e0 0x7fa0fc0cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37450/0 (socket says 172.21.15.92:37450) 2023-06-09T04:35:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0faffd700 1 -- 172.21.15.92:0/65272521 learned_addr learned my addr 172.21.15.92:0/65272521 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0fa7fc700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 0x7fa0fc0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0fb7fe700 1 --2- 172.21.15.92:0/65272521 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc06aec0 0x7fa0fc06b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707993097 0 0) 0x7fa0fc0ce720 con 0x7fa0fc15c5e0 2023-06-09T04:35:59.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e40035b0 con 0x7fa0fc15c5e0 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162744091 0 0) 0x7fa0e40035b0 con 0x7fa0fc15c5e0 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 msgr2=0x7fa0fc0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0f9ffb700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 0x7fa0fc0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc06aec0 msgr2=0x7fa0fc06b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.582+0000 7fa0f9ffb700 1 --2- 172.21.15.92:0/65272521 >> v2:172.21.15.123:3300/0 conn(0x7fa0fc06aec0 0x7fa0fc06b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0cefc0 con 0x7fa0fc15c5e0 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fa0f00017f0 con 0x7fa0fc15c5e0 2023-06-09T04:35:59.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 >> v1:172.21.15.92:6789/0 conn(0x7fa0fc15c5e0 legacy=0x7fa0fc0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0e4003a00 0x7fa0e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 0x7fa0e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4004d90 0x7fa0e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0cea00 con 0x7fa0e4004d90 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0cece0 con 0x7fa0e4003a00 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e400ba10 con 0x7fa0fc0d0370 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fb7fe700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4004d90 0x7fa0e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0faffd700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0e4003a00 0x7fa0e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fa7fc700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 0x7fa0e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fb7fe700 1 -- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 msgr2=0x7fa0e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fb7fe700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc0d0370 0x7fa0e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fb7fe700 1 -- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0e4003a00 msgr2=0x7fa0e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fb7fe700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0e4003a00 0x7fa0e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fb7fe700 1 -- 172.21.15.92:0/65272521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40035b0 con 0x7fa0e4004d90 2023-06-09T04:35:59.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0fb7fe700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4004d90 0x7fa0e4005200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009f70 tx=0x7fa0e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e165269fea26dee server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4003a20 con 0x7fa0e4004d90 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f400f040 con 0x7fa0e4004d90 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.583+0000 7fa0f9ffb700 1 -- 172.21.15.92:0/65272521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4003dc0 con 0x7fa0e4004d90 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.584+0000 7fa1012c9700 1 -- 172.21.15.92:0/65272521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4004d90 msgr2=0x7fa0e4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.584+0000 7fa1012c9700 1 --2- 172.21.15.92:0/65272521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0e4004d90 0x7fa0e4005200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009f70 tx=0x7fa0e4005880 comp rx=0 tx=0).stop 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.584+0000 7fa0fb7fe700 1 -- 172.21.15.92:0/65272521 reap_dead start 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.584+0000 7fa1012c9700 1 -- 172.21.15.92:0/65272521 shutdown_connections 2023-06-09T04:35:59.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.584+0000 7fa1012c9700 1 -- 172.21.15.92:0/65272521 >> 172.21.15.92:0/65272521 conn(0x7fa0fc066370 msgr2=0x7fa0fc099d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:59.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.584+0000 7fa1012c9700 1 -- 172.21.15.92:0/65272521 shutdown_connections 2023-06-09T04:35:59.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.584+0000 7fa1012c9700 1 -- 172.21.15.92:0/65272521 wait complete. 2023-06-09T04:35:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.585+0000 7fa1012c9700 1 Processor -- start 2023-06-09T04:35:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.585+0000 7fa1012c9700 1 -- start start 2023-06-09T04:35:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa1012c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0fc0c6d70 0x7fa0fc0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa1012c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc0c7140 0x7fa0fc1f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa1012c9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc1f23d0 0x7fa0fc15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa1012c9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0ae440 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa1012c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc1f2910 con 0x7fa0fc0c7140 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa1012c9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc1f2bf0 con 0x7fa0fc1f23d0 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0fc0c6d70 0x7fa0fc0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0fc0c6d70 0x7fa0fc0adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39130/0 (socket says 172.21.15.92:39130) 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 -- 172.21.15.92:0/940206753 learned_addr learned my addr 172.21.15.92:0/940206753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0fa7fc700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc0c7140 0x7fa0fc1f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0fb7fe700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc1f23d0 0x7fa0fc15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 -- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc1f23d0 msgr2=0x7fa0fc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc1f23d0 0x7fa0fc15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 -- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc0c7140 msgr2=0x7fa0fc1f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc0c7140 0x7fa0fc1f1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0faffd700 1 -- 172.21.15.92:0/940206753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0ad700 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.586+0000 7fa0fb7fe700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc1f23d0 0x7fa0fc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.587+0000 7fa0faffd700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0fc0c6d70 0x7fa0fc0adf00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000ef0 tx=0x7fa0f0001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.587+0000 7fa0e37fe700 1 -- 172.21.15.92:0/940206753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00073a0 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.587+0000 7fa0e37fe700 1 -- 172.21.15.92:0/940206753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f0007500 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.587+0000 7fa0e37fe700 1 -- 172.21.15.92:0/940206753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0005a50 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.587+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc0ad9e0 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.587+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc0afa20 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.588+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc1f2e10 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.588+0000 7fa0e37fe700 1 -- 172.21.15.92:0/940206753 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa0f0005bb0 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.588+0000 7fa0e37fe700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0e806ace0 0x7fa0e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.589+0000 7fa0fa7fc700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0e806ace0 0x7fa0e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.589+0000 7fa0e37fe700 1 -- 172.21.15.92:0/940206753 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa0f00895f0 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.589+0000 7fa0fa7fc700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0e806ace0 0x7fa0e806d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0ce900 tx=0x7fa0ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.593+0000 7fa0e37fe700 1 -- 172.21.15.92:0/940206753 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa0f0054660 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.778+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0fc04e8e0 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.778+0000 7fa0e37fe700 1 -- 172.21.15.92:0/940206753 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0f0005e60 con 0x7fa0fc0c6d70 2023-06-09T04:35:59.779 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:35:59.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0e806ace0 msgr2=0x7fa0e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0e806ace0 0x7fa0e806d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0ce900 tx=0x7fa0ec008040 comp rx=0 tx=0).stop 2023-06-09T04:35:59.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0fc0c6d70 msgr2=0x7fa0fc0adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0fc0c6d70 0x7fa0fc0adf00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0000ef0 tx=0x7fa0f0001890 comp rx=0 tx=0).stop 2023-06-09T04:35:59.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 shutdown_connections 2023-06-09T04:35:59.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0e806ace0 0x7fa0e806d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0fc1f23d0 0x7fa0fc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0fc0c6d70 0x7fa0fc0adf00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 --2- 172.21.15.92:0/940206753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0fc0c7140 0x7fa0fc1f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 >> 172.21.15.92:0/940206753 conn(0x7fa0fc066370 msgr2=0x7fa0fc15b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:59.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 shutdown_connections 2023-06-09T04:35:59.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.780+0000 7fa1012c9700 1 -- 172.21.15.92:0/940206753 wait complete. 2023-06-09T04:35:59.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:35:59.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.914+0000 7f344aeab700 1 Processor -- start 2023-06-09T04:35:59.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.914+0000 7f344aeab700 1 -- start start 2023-06-09T04:35:59.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.914+0000 7f344aeab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34440ac400 0x7f34440ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.914+0000 7f344aeab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34440ce730 con 0x7f34440cf010 2023-06-09T04:35:59.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.914+0000 7f344aeab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344406aaf0 con 0x7f344415c7d0 2023-06-09T04:35:59.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.914+0000 7f344aeab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344415cba0 con 0x7f34440ac400 2023-06-09T04:35:59.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.914+0000 7f3443fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f34440cf010 0x7f344406e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37470/0 (socket says 172.21.15.92:37470) 2023-06-09T04:35:59.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3443fff700 1 -- 172.21.15.92:0/805993773 learned_addr learned my addr 172.21.15.92:0/805993773 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3449448700 1 --2- 172.21.15.92:0/805993773 >> v2:172.21.15.123:3300/0 conn(0x7f34440ac400 0x7f34440ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169421311 0 0) 0x7f3430001110 con 0x7f34440cf010 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34340035b0 con 0x7f34440cf010 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3147310502 0 0) 0x7f34340035b0 con 0x7f34440cf010 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 >> v2:172.21.15.123:3300/0 conn(0x7f34440ac400 msgr2=0x7f34440ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 --2- 172.21.15.92:0/805993773 >> v2:172.21.15.123:3300/0 conn(0x7f34440ac400 0x7f34440ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 >> v1:172.21.15.123:6789/0 conn(0x7f344415c7d0 legacy=0x7f34440ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3444069ee0 con 0x7f34440cf010 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f3430001800 con 0x7f34440cf010 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 >> v1:172.21.15.92:6789/0 conn(0x7f34440cf010 legacy=0x7f344406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3434003a00 0x7f3434003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f3434004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3434005200 0x7f343400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344415cba0 con 0x7f3434004850 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343400c110 con 0x7f3434003a00 2023-06-09T04:35:59.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343400c2d0 con 0x7f3434005200 2023-06-09T04:35:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3443fff700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f3434004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3448c47700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3434003a00 0x7f3434003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3449448700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3434005200 0x7f343400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3443fff700 1 -- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3434005200 msgr2=0x7f343400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3443fff700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3434005200 0x7f343400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3443fff700 1 -- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3434003a00 msgr2=0x7f3434003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:35:59.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3443fff700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3434003a00 0x7f3434003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3449448700 1 -- 172.21.15.92:0/805993773 reap_dead start 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3443fff700 1 -- 172.21.15.92:0/805993773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34340035b0 con 0x7f3434004850 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.915+0000 7f3448c47700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3434003a00 0x7f3434003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f3443fff700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f3434004cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3430007fd0 tx=0x7f3434005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80b848218259d8d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430007d90 con 0x7f3434004850 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343000cd00 con 0x7f3434004850 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f34437fe700 1 -- 172.21.15.92:0/805993773 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430005b80 con 0x7f3434004850 2023-06-09T04:35:59.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f344aeab700 1 -- 172.21.15.92:0/805993773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 msgr2=0x7f3434004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f344aeab700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f3434004cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3430007fd0 tx=0x7f3434005880 comp rx=0 tx=0).stop 2023-06-09T04:35:59.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f344aeab700 1 -- 172.21.15.92:0/805993773 shutdown_connections 2023-06-09T04:35:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f344aeab700 1 --2- 172.21.15.92:0/805993773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f3434004cc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f344aeab700 1 -- 172.21.15.92:0/805993773 >> 172.21.15.92:0/805993773 conn(0x7f3444098c10 msgr2=0x7f34440cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:35:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f344aeab700 1 -- 172.21.15.92:0/805993773 shutdown_connections 2023-06-09T04:35:59.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.916+0000 7f344aeab700 1 -- 172.21.15.92:0/805993773 wait complete. 2023-06-09T04:35:59.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.918+0000 7f344aeab700 1 Processor -- start 2023-06-09T04:35:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.918+0000 7f344aeab700 1 -- start start 2023-06-09T04:35:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f344aeab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f344409ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f344aeab700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34441f5e40 0x7f3444149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f344aeab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34441f51c0 0x7f34441f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f344aeab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34441f5ba0 con 0x7f3434004850 2023-06-09T04:35:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f344aeab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344409a460 con 0x7f34441f51c0 2023-06-09T04:35:59.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f344aeab700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344409afa0 con 0x7f34441f5e40 2023-06-09T04:35:59.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f344409ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f344409ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39150/0 (socket says 172.21.15.92:39150) 2023-06-09T04:35:59.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 -- 172.21.15.92:0/3470371385 learned_addr learned my addr 172.21.15.92:0/3470371385 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:35:59.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3443fff700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34441f5e40 0x7f3444149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3449448700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34441f51c0 0x7f34441f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 -- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34441f5e40 msgr2=0x7f3444149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34441f5e40 0x7f3444149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 -- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34441f51c0 msgr2=0x7f34441f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34441f51c0 0x7f34441f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3448c47700 1 -- 172.21.15.92:0/3470371385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344409a670 con 0x7f3434004850 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3443fff700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34441f5e40 0x7f3444149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.919+0000 7f3449448700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34441f51c0 0x7f34441f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.920+0000 7f3448c47700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f344409ce50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f343800e460 tx=0x7f343800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.920+0000 7f34417fa700 1 -- 172.21.15.92:0/3470371385 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34380052e0 con 0x7f3434004850 2023-06-09T04:35:59.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.920+0000 7f34417fa700 1 -- 172.21.15.92:0/3470371385 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3438005440 con 0x7f3434004850 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.920+0000 7f34417fa700 1 -- 172.21.15.92:0/3470371385 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438003910 con 0x7f3434004850 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.920+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344409a950 con 0x7f3434004850 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.920+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344409d390 con 0x7f3434004850 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.921+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34440740e0 con 0x7f3434004850 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.921+0000 7f34417fa700 1 -- 172.21.15.92:0/3470371385 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3438003a70 con 0x7f3434004850 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.922+0000 7f34417fa700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f342806ace0 0x7f342806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.922+0000 7f34417fa700 1 -- 172.21.15.92:0/3470371385 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f34380888e0 con 0x7f3434004850 2023-06-09T04:35:59.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.922+0000 7f3443fff700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f342806ace0 0x7f342806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:35:59.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.922+0000 7f3443fff700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f342806ace0 0x7f342806d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3430015040 tx=0x7f3430007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:35:59.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:35:59.926+0000 7f34417fa700 1 -- 172.21.15.92:0/3470371385 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3438053950 con 0x7f3434004850 2023-06-09T04:36:00.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.155+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34440b1e30 con 0x7f3434004850 2023-06-09T04:36:00.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.155+0000 7f34417fa700 1 -- 172.21.15.92:0/3470371385 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f3438056f70 con 0x7f3434004850 2023-06-09T04:36:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:00.156 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:36:00.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f342806ace0 msgr2=0x7f342806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f342806ace0 0x7f342806d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3430015040 tx=0x7f3430007000 comp rx=0 tx=0).stop 2023-06-09T04:36:00.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 msgr2=0x7f344409ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f344409ce50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f343800e460 tx=0x7f343800e820 comp rx=0 tx=0).stop 2023-06-09T04:36:00.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 shutdown_connections 2023-06-09T04:36:00.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f342806ace0 0x7f342806d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34441f51c0 0x7f34441f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3434004850 0x7f344409ce50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 --2- 172.21.15.92:0/3470371385 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34441f5e40 0x7f3444149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 >> 172.21.15.92:0/3470371385 conn(0x7f3444098c10 msgr2=0x7f344406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:00.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 shutdown_connections 2023-06-09T04:36:00.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.157+0000 7f344aeab700 1 -- 172.21.15.92:0/3470371385 wait complete. 2023-06-09T04:36:00.159 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:36:00.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:36:00.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.292+0000 7fee52da3700 1 Processor -- start 2023-06-09T04:36:00.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.292+0000 7fee52da3700 1 -- start start 2023-06-09T04:36:00.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee52da3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a500 0x7fee4c0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee52da3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee4c15a8d0 0x7fee4c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee52da3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c0ac400 con 0x7fee4c0cbf50 2023-06-09T04:36:00.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee52da3700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c0cd7e0 con 0x7fee4c15a8d0 2023-06-09T04:36:00.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee52da3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0cdda0 con 0x7fee4c15a500 2023-06-09T04:36:00.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee4bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee4c15a8d0 0x7fee4c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee50b3f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a500 0x7fee4c0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee4bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee4c15a8d0 0x7fee4c069db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:59566/0 (socket says 172.21.15.92:59566) 2023-06-09T04:36:00.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee4bfff700 1 -- 172.21.15.92:0/121556917 learned_addr learned my addr 172.21.15.92:0/121556917 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:00.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee4bfff700 1 -- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a500 msgr2=0x7fee4c0cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:00.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee4bfff700 1 --2- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a500 0x7fee4c0cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee4bfff700 1 -- 172.21.15.92:0/121556917 >> v1:172.21.15.123:6789/0 conn(0x7fee4c0cbf50 legacy=0x7fee4c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee4bfff700 1 -- 172.21.15.92:0/121556917 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0aeca0 con 0x7fee4c15a8d0 2023-06-09T04:36:00.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.293+0000 7fee50b3f700 1 --2- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a500 0x7fee4c0cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:36:00.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee4bfff700 1 --2- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a8d0 0x7fee4c069db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee380098f0 tx=0x7fee38009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd71bfe078217da server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:00.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee4b7fe700 1 -- 172.21.15.92:0/121556917 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38011830 con 0x7fee4c15a8d0 2023-06-09T04:36:00.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee4b7fe700 1 -- 172.21.15.92:0/121556917 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee38011990 con 0x7fee4c15a8d0 2023-06-09T04:36:00.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee4b7fe700 1 -- 172.21.15.92:0/121556917 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38011d20 con 0x7fee4c15a8d0 2023-06-09T04:36:00.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 -- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a8d0 msgr2=0x7fee4c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 --2- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a8d0 0x7fee4c069db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fee380098f0 tx=0x7fee38009c00 comp rx=0 tx=0).stop 2023-06-09T04:36:00.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 -- 172.21.15.92:0/121556917 shutdown_connections 2023-06-09T04:36:00.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 --2- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a500 0x7fee4c0cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 --2- 172.21.15.92:0/121556917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a8d0 0x7fee4c069db0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 -- 172.21.15.92:0/121556917 >> 172.21.15.92:0/121556917 conn(0x7fee4c098c10 msgr2=0x7fee4c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 -- 172.21.15.92:0/121556917 shutdown_connections 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.294+0000 7fee52da3700 1 -- 172.21.15.92:0/121556917 wait complete. 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.295+0000 7fee52da3700 1 Processor -- start 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.296+0000 7fee52da3700 1 -- start start 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.296+0000 7fee52da3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee4c0cbf50 0x7fee4c14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.296+0000 7fee52da3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a500 0x7fee4c0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.296+0000 7fee52da3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a8d0 0x7fee4c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.296+0000 7fee52da3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1f5da0 con 0x7fee4c0cbf50 2023-06-09T04:36:00.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.296+0000 7fee52da3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1f51b0 con 0x7fee4c15a500 2023-06-09T04:36:00.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.296+0000 7fee52da3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1f5470 con 0x7fee4c15a8d0 2023-06-09T04:36:00.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee4c0cbf50 0x7fee4c14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee4c0cbf50 0x7fee4c14dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39162/0 (socket says 172.21.15.92:39162) 2023-06-09T04:36:00.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 -- 172.21.15.92:0/1421262504 learned_addr learned my addr 172.21.15.92:0/1421262504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee4bfff700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a500 0x7fee4c0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee51340700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a8d0 0x7fee4c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 -- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a8d0 msgr2=0x7fee4c0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a8d0 0x7fee4c0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 -- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a500 msgr2=0x7fee4c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a500 0x7fee4c0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 -- 172.21.15.92:0/1421262504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40009430 con 0x7fee4c0cbf50 2023-06-09T04:36:00.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee50b3f700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee4c0cbf50 0x7fee4c14dd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fee4000ebe0 tx=0x7fee4000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:00.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee497fa700 1 -- 172.21.15.92:0/1421262504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40005a20 con 0x7fee4c0cbf50 2023-06-09T04:36:00.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee497fa700 1 -- 172.21.15.92:0/1421262504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee40005b80 con 0x7fee4c0cbf50 2023-06-09T04:36:00.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee38009550 con 0x7fee4c0cbf50 2023-06-09T04:36:00.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee497fa700 1 -- 172.21.15.92:0/1421262504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4000c850 con 0x7fee4c0cbf50 2023-06-09T04:36:00.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.297+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c1f5a30 con 0x7fee4c0cbf50 2023-06-09T04:36:00.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.298+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c0740e0 con 0x7fee4c0cbf50 2023-06-09T04:36:00.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.299+0000 7fee497fa700 1 -- 172.21.15.92:0/1421262504 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee4000c9b0 con 0x7fee4c0cbf50 2023-06-09T04:36:00.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.299+0000 7fee497fa700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fee3006ace0 0x7fee3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.299+0000 7fee497fa700 1 -- 172.21.15.92:0/1421262504 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fee40089740 con 0x7fee4c0cbf50 2023-06-09T04:36:00.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.299+0000 7fee4bfff700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fee3006ace0 0x7fee3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.299+0000 7fee4bfff700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fee3006ace0 0x7fee3006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee4c15aca0 tx=0x7fee38008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:00.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.303+0000 7fee497fa700 1 -- 172.21.15.92:0/1421262504 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fee400547b0 con 0x7fee4c0cbf50 2023-06-09T04:36:00.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.492+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee4c0ad690 con 0x7fee4c0cbf50 2023-06-09T04:36:00.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.493+0000 7fee497fa700 1 -- 172.21.15.92:0/1421262504 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fee40057dd0 con 0x7fee4c0cbf50 2023-06-09T04:36:00.494 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:00.494 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:59.229550+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:36:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.495+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fee3006ace0 msgr2=0x7fee3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.495+0000 7fee52da3700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fee3006ace0 0x7fee3006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee4c15aca0 tx=0x7fee38008030 comp rx=0 tx=0).stop 2023-06-09T04:36:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.495+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee4c0cbf50 msgr2=0x7fee4c14dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.495+0000 7fee52da3700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee4c0cbf50 0x7fee4c14dd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fee4000ebe0 tx=0x7fee4000eef0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 shutdown_connections 2023-06-09T04:36:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fee3006ace0 0x7fee3006d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee4c15a8d0 0x7fee4c0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee4c0cbf50 0x7fee4c14dd60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 --2- 172.21.15.92:0/1421262504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c15a500 0x7fee4c0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 >> 172.21.15.92:0/1421262504 conn(0x7fee4c098c10 msgr2=0x7fee4c0ac590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 shutdown_connections 2023-06-09T04:36:00.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.496+0000 7fee52da3700 1 -- 172.21.15.92:0/1421262504 wait complete. 2023-06-09T04:36:00.508 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-09T04:36:00.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:36:00.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.627+0000 7f05a14b0700 1 Processor -- start 2023-06-09T04:36:00.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.628+0000 7f05a14b0700 1 -- start start 2023-06-09T04:36:00.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.628+0000 7f05a14b0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f059c0b2070 0x7f059c0ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.628+0000 7f05a14b0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c0d17c0 con 0x7f059c0cd1a0 2023-06-09T04:36:00.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.628+0000 7f05a14b0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c0d0720 con 0x7f059c0cfc40 2023-06-09T04:36:00.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.628+0000 7f05a14b0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0d0a00 con 0x7f059c0b2070 2023-06-09T04:36:00.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.628+0000 7f059a7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f059c0cd1a0 0x7f059c0cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37504/0 (socket says 172.21.15.92:37504) 2023-06-09T04:36:00.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.628+0000 7f059a7fc700 1 -- 172.21.15.92:0/4009863731 learned_addr learned my addr 172.21.15.92:0/4009863731 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:00.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059affd700 1 --2- 172.21.15.92:0/4009863731 >> v2:172.21.15.123:3300/0 conn(0x7f059c0b2070 0x7f059c0ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202218023 0 0) 0x7f0584001110 con 0x7f059c0cd1a0 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05880035b0 con 0x7f059c0cd1a0 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3259651627 0 0) 0x7f05880035b0 con 0x7f059c0cd1a0 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 >> v2:172.21.15.123:3300/0 conn(0x7f059c0b2070 msgr2=0x7f059c0ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 --2- 172.21.15.92:0/4009863731 >> v2:172.21.15.123:3300/0 conn(0x7f059c0b2070 0x7f059c0ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 >> v1:172.21.15.123:6789/0 conn(0x7f059c0cfc40 legacy=0x7f059c0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0d0ce0 con 0x7f059c0cd1a0 2023-06-09T04:36:00.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f0584001800 con 0x7f059c0cd1a0 2023-06-09T04:36:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059affd700 1 --2- 172.21.15.92:0/4009863731 >> v2:172.21.15.123:3300/0 conn(0x7f059c0b2070 0x7f059c0ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 >> v1:172.21.15.92:6789/0 conn(0x7f059c0cd1a0 legacy=0x7f059c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0588003a00 0x7f0588003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f0588004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0588005320 0x7f058800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0d0a00 con 0x7f0588004850 2023-06-09T04:36:00.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058800c1e0 con 0x7f0588005320 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058800c3a0 con 0x7f0588003a00 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059a7fc700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f0588004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059affd700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0588003a00 0x7f0588003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059b7fe700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0588005320 0x7f058800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059a7fc700 1 -- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0588003a00 msgr2=0x7f0588003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059a7fc700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0588003a00 0x7f0588003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059a7fc700 1 -- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0588005320 msgr2=0x7f058800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:00.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059a7fc700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0588005320 0x7f058800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059a7fc700 1 -- 172.21.15.92:0/4009863731 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05880035b0 con 0x7f0588004850 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059b7fe700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0588005320 0x7f058800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.629+0000 7f059b7fe700 1 -- 172.21.15.92:0/4009863731 reap_dead start 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f059a7fc700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f0588004cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f058400c210 tx=0x7f0588005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d052ae5c8723226a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584007db0 con 0x7f0588004850 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058400cdd0 con 0x7f0588004850 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f0599ffb700 1 -- 172.21.15.92:0/4009863731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584005ba0 con 0x7f0588004850 2023-06-09T04:36:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f05a14b0700 1 -- 172.21.15.92:0/4009863731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 msgr2=0x7f0588004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f05a14b0700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f0588004cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f058400c210 tx=0x7f0588005880 comp rx=0 tx=0).stop 2023-06-09T04:36:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f05a14b0700 1 -- 172.21.15.92:0/4009863731 shutdown_connections 2023-06-09T04:36:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f05a14b0700 1 --2- 172.21.15.92:0/4009863731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f0588004cc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f05a14b0700 1 -- 172.21.15.92:0/4009863731 >> 172.21.15.92:0/4009863731 conn(0x7f059c065a50 msgr2=0x7f059c065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f05a14b0700 1 -- 172.21.15.92:0/4009863731 shutdown_connections 2023-06-09T04:36:00.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.630+0000 7f05a14b0700 1 -- 172.21.15.92:0/4009863731 wait complete. 2023-06-09T04:36:00.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 Processor -- start 2023-06-09T04:36:00.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 -- start start 2023-06-09T04:36:00.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f059c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f059c0c78a0 0x7f059c153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059c15de90 0x7f059c15e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c15e850 con 0x7f0588004850 2023-06-09T04:36:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c8970 con 0x7f059c0c78a0 2023-06-09T04:36:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f05a14b0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c7d60 con 0x7f059c15de90 2023-06-09T04:36:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f059affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f059c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f059affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f059c0cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39182/0 (socket says 172.21.15.92:39182) 2023-06-09T04:36:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.632+0000 7f059affd700 1 -- 172.21.15.92:0/329890176 learned_addr learned my addr 172.21.15.92:0/329890176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f059a7fc700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f059c0c78a0 0x7f059c153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f059affd700 1 -- 172.21.15.92:0/329890176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059c15de90 msgr2=0x7f059c15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f059affd700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059c15de90 0x7f059c15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f059affd700 1 -- 172.21.15.92:0/329890176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f059c0c78a0 msgr2=0x7f059c153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f059affd700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f059c0c78a0 0x7f059c153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f059affd700 1 -- 172.21.15.92:0/329890176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0c8040 con 0x7f0588004850 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f059affd700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f059c0cc830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f058c011900 tx=0x7f058c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:00.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f05937fe700 1 -- 172.21.15.92:0/329890176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c004230 con 0x7f0588004850 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f05937fe700 1 -- 172.21.15.92:0/329890176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c004390 con 0x7f0588004850 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f05937fe700 1 -- 172.21.15.92:0/329890176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0036a0 con 0x7f0588004850 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0c8320 con 0x7f0588004850 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.633+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c0ccd70 con 0x7f0588004850 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.634+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c0740e0 con 0x7f0588004850 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.635+0000 7f05937fe700 1 -- 172.21.15.92:0/329890176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f058c003800 con 0x7f0588004850 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.635+0000 7f05937fe700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f057c06ace0 0x7f057c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.635+0000 7f05937fe700 1 -- 172.21.15.92:0/329890176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f058c0899f0 con 0x7f0588004850 2023-06-09T04:36:00.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.635+0000 7f059a7fc700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f057c06ace0 0x7f057c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:00.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.636+0000 7f059a7fc700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f057c06ace0 0x7f057c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0584015040 tx=0x7f0584007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:00.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.639+0000 7f05937fe700 1 -- 172.21.15.92:0/329890176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f058c0585d0 con 0x7f0588004850 2023-06-09T04:36:00.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.866+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f059c0b70b0 con 0x7f0588004850 2023-06-09T04:36:00.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.867+0000 7f05937fe700 1 -- 172.21.15.92:0/329890176 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f058c058160 con 0x7f0588004850 2023-06-09T04:36:00.867 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:00.867 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:36:00.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f057c06ace0 msgr2=0x7f057c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f057c06ace0 0x7f057c06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0584015040 tx=0x7f0584007000 comp rx=0 tx=0).stop 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 msgr2=0x7f059c0cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f059c0cc830 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f058c011900 tx=0x7f058c011cc0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 shutdown_connections 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f057c06ace0 0x7f057c06d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059c15de90 0x7f059c15e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0588004850 0x7f059c0cc830 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 --2- 172.21.15.92:0/329890176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f059c0c78a0 0x7f059c153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 >> 172.21.15.92:0/329890176 conn(0x7f059c065a50 msgr2=0x7f059c0b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 shutdown_connections 2023-06-09T04:36:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:00.869+0000 7f05a14b0700 1 -- 172.21.15.92:0/329890176 wait complete. 2023-06-09T04:36:00.871 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:36:00.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:36:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.000+0000 7f4897ce3700 1 Processor -- start 2023-06-09T04:36:01.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.001+0000 7f4897ce3700 1 -- start start 2023-06-09T04:36:01.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.001+0000 7f4897ce3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f48900ae630 0x7f48900aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.001+0000 7f4897ce3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900cd7e0 0x7f48900cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4897ce3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48900adde0 con 0x7f48900b1360 2023-06-09T04:36:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4897ce3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48900ae270 con 0x7f48900ae630 2023-06-09T04:36:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4897ce3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48900af0f0 con 0x7f48900cd7e0 2023-06-09T04:36:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f489527e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f48900ae630 0x7f48900aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4895a7f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f48900b1360 0x7f48900b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37522/0 (socket says 172.21.15.92:37522) 2023-06-09T04:36:01.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4895a7f700 1 -- 172.21.15.92:0/2013079125 learned_addr learned my addr 172.21.15.92:0/2013079125 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4896280700 1 --2- 172.21.15.92:0/2013079125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900cd7e0 0x7f48900cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f489527e700 1 -- 172.21.15.92:0/2013079125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900cd7e0 msgr2=0x7f48900cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f489527e700 1 --2- 172.21.15.92:0/2013079125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900cd7e0 0x7f48900cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f489527e700 1 -- 172.21.15.92:0/2013079125 >> v1:172.21.15.92:6789/0 conn(0x7f48900b1360 legacy=0x7f48900b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4894a7d700 1 -- 172.21.15.92:0/2013079125 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263036451 0 0) 0x7f48900adde0 con 0x7f48900b1360 2023-06-09T04:36:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f489527e700 1 -- 172.21.15.92:0/2013079125 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48900ad4b0 con 0x7f48900ae630 2023-06-09T04:36:01.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4896280700 1 --2- 172.21.15.92:0/2013079125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900cd7e0 0x7f48900cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f489527e700 1 --2- 172.21.15.92:0/2013079125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48900aeaa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f488c00bae0 tx=0x7f488c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520b1e81e7d4b255 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4894a7d700 1 -- 172.21.15.92:0/2013079125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00dd30 con 0x7f48900ae630 2023-06-09T04:36:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4894a7d700 1 -- 172.21.15.92:0/2013079125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c00de90 con 0x7f48900ae630 2023-06-09T04:36:01.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.002+0000 7f4894a7d700 1 -- 172.21.15.92:0/2013079125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c010690 con 0x7f48900ae630 2023-06-09T04:36:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 -- 172.21.15.92:0/2013079125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 msgr2=0x7f48900aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 --2- 172.21.15.92:0/2013079125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48900aeaa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f488c00bae0 tx=0x7f488c00bdf0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 -- 172.21.15.92:0/2013079125 shutdown_connections 2023-06-09T04:36:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 --2- 172.21.15.92:0/2013079125 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900cd7e0 0x7f48900cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 --2- 172.21.15.92:0/2013079125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48900aeaa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 -- 172.21.15.92:0/2013079125 >> 172.21.15.92:0/2013079125 conn(0x7f4890098c10 msgr2=0x7f48900cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 -- 172.21.15.92:0/2013079125 shutdown_connections 2023-06-09T04:36:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.003+0000 7f4897ce3700 1 -- 172.21.15.92:0/2013079125 wait complete. 2023-06-09T04:36:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.004+0000 7f4897ce3700 1 Processor -- start 2023-06-09T04:36:01.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.004+0000 7f4897ce3700 1 -- start start 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48901f1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b1360 0x7f48900c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48900cd7e0 0x7f48900c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48901f2050 con 0x7f48900ae630 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48901f1a90 con 0x7f48900cd7e0 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48901f1d70 con 0x7f48900b1360 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48901f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48901f1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39196/0 (socket says 172.21.15.92:39196) 2023-06-09T04:36:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 -- 172.21.15.92:0/3351112410 learned_addr learned my addr 172.21.15.92:0/3351112410 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f489527e700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b1360 0x7f48900c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4896280700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48900cd7e0 0x7f48900c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 -- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b1360 msgr2=0x7f48900c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b1360 0x7f48900c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 -- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48900cd7e0 msgr2=0x7f48900c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48900cd7e0 0x7f48900c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 -- 172.21.15.92:0/3351112410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c00b740 con 0x7f48900ae630 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4895a7f700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48901f1140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f488000c700 tx=0x7f488000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f48867fc700 1 -- 172.21.15.92:0/3351112410 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48800074e0 con 0x7f48900ae630 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f48867fc700 1 -- 172.21.15.92:0/3351112410 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880007640 con 0x7f48900ae630 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48900cb390 con 0x7f48900ae630 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f48867fc700 1 -- 172.21.15.92:0/3351112410 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880005a30 con 0x7f48900ae630 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.005+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48901f1680 con 0x7f48900ae630 2023-06-09T04:36:01.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.006+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48900cb8b0 con 0x7f48900ae630 2023-06-09T04:36:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.007+0000 7f48867fc700 1 -- 172.21.15.92:0/3351112410 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4880005b90 con 0x7f48900ae630 2023-06-09T04:36:01.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.007+0000 7f48867fc700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f487406ace0 0x7f487406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.007+0000 7f48867fc700 1 -- 172.21.15.92:0/3351112410 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f48800888a0 con 0x7f48900ae630 2023-06-09T04:36:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.007+0000 7f489527e700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f487406ace0 0x7f487406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.008+0000 7f489527e700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f487406ace0 0x7f487406d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f488c00be30 tx=0x7f488c013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.011+0000 7f48867fc700 1 -- 172.21.15.92:0/3351112410 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f48900cb8b0 con 0x7f48900ae630 2023-06-09T04:36:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.242+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48900cb8b0 con 0x7f48900ae630 2023-06-09T04:36:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.244+0000 7f48867fc700 1 -- 172.21.15.92:0/3351112410 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4668 (secure 0 0 0) 0x7f4880017090 con 0x7f48900ae630 2023-06-09T04:36:01.245 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:01.245 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/3018963981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3018963981},{"type":"v1","addr":"172.21.15.123:6834","nonce":3018963981}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3744361359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3744361359},{"type":"v1","addr":"172.21.15.92:6835","nonce":3744361359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1756419566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1756419566},{"type":"v1","addr":"172.21.15.92:6837","nonce":1756419566}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:35:17.412184+0000","modified":"2023-06-09T04:35:20.426961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/218299678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":218299678},{"type":"v1","addr":"172.21.15.123:6835","nonce":218299678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T04:36:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f487406ace0 msgr2=0x7f487406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f487406ace0 0x7f487406d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f488c00be30 tx=0x7f488c013000 comp rx=0 tx=0).stop 2023-06-09T04:36:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 msgr2=0x7f48901f1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48901f1140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f488000c700 tx=0x7f488000cac0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 shutdown_connections 2023-06-09T04:36:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f487406ace0 0x7f487406d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48900cd7e0 0x7f48900c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48900ae630 0x7f48901f1140 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 --2- 172.21.15.92:0/3351112410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f48900b1360 0x7f48900c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.246+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 >> 172.21.15.92:0/3351112410 conn(0x7f4890098c10 msgr2=0x7f489006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.247+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 shutdown_connections 2023-06-09T04:36:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.247+0000 7f4897ce3700 1 -- 172.21.15.92:0/3351112410 wait complete. 2023-06-09T04:36:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 8 2023-06-09T04:36:01.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:36:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.378+0000 7fa11185a700 1 Processor -- start 2023-06-09T04:36:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.378+0000 7fa11185a700 1 -- start start 2023-06-09T04:36:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa11185a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa10c0b2070 0x7fa10c0ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa11185a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa10c0cd1a0 0x7fa10c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa11185a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cfa70 0x7fa10c0d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa11185a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10c0cff10 con 0x7fa10c0b2070 2023-06-09T04:36:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa11185a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10c0d17c0 con 0x7fa10c0cd1a0 2023-06-09T04:36:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa11185a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0d0720 con 0x7fa10c0cfa70 2023-06-09T04:36:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa10c0b2070 0x7fa10c0ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa10c0b2070 0x7fa10c0ced00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39204/0 (socket says 172.21.15.92:39204) 2023-06-09T04:36:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 -- 172.21.15.92:0/395804652 learned_addr learned my addr 172.21.15.92:0/395804652 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10a7fc700 1 --2- 172.21.15.92:0/395804652 >> v2:172.21.15.123:3300/0 conn(0x7fa10c0cd1a0 0x7fa10c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 -- 172.21.15.92:0/395804652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cfa70 msgr2=0x7fa10c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 --2- 172.21.15.92:0/395804652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cfa70 0x7fa10c0d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 -- 172.21.15.92:0/395804652 >> v2:172.21.15.123:3300/0 conn(0x7fa10c0cd1a0 msgr2=0x7fa10c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 --2- 172.21.15.92:0/395804652 >> v2:172.21.15.123:3300/0 conn(0x7fa10c0cd1a0 0x7fa10c0cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.379+0000 7fa10affd700 1 -- 172.21.15.92:0/395804652 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0d0a00 con 0x7fa10c0b2070 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa10affd700 1 --2- 172.21.15.92:0/395804652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c0ced00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0ceaf0 tx=0x7fa100009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9989114285766dc1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa109ffb700 1 -- 172.21.15.92:0/395804652 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000b870 con 0x7fa10c0b2070 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa109ffb700 1 -- 172.21.15.92:0/395804652 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10000b9d0 con 0x7fa10c0b2070 2023-06-09T04:36:01.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa109ffb700 1 -- 172.21.15.92:0/395804652 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10000bd60 con 0x7fa10c0b2070 2023-06-09T04:36:01.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 -- 172.21.15.92:0/395804652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 msgr2=0x7fa10c0ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 --2- 172.21.15.92:0/395804652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c0ced00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0ceaf0 tx=0x7fa100009bd0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 -- 172.21.15.92:0/395804652 shutdown_connections 2023-06-09T04:36:01.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 --2- 172.21.15.92:0/395804652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cfa70 0x7fa10c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 --2- 172.21.15.92:0/395804652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c0ced00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 --2- 172.21.15.92:0/395804652 >> v2:172.21.15.123:3300/0 conn(0x7fa10c0cd1a0 0x7fa10c0cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 -- 172.21.15.92:0/395804652 >> 172.21.15.92:0/395804652 conn(0x7fa10c065a50 msgr2=0x7fa10c065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 -- 172.21.15.92:0/395804652 shutdown_connections 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.380+0000 7fa11185a700 1 -- 172.21.15.92:0/395804652 wait complete. 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 Processor -- start 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 -- start start 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cd1a0 0x7fa10c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10c0cfa70 0x7fa10c15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0c9c20 con 0x7fa10c0b2070 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0c9ec0 con 0x7fa10c0cfa70 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa11185a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0ca650 con 0x7fa10c0cd1a0 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.382+0000 7fa10affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39210/0 (socket says 172.21.15.92:39210) 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 -- 172.21.15.92:0/47842699 learned_addr learned my addr 172.21.15.92:0/47842699 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10b7fe700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10c0cfa70 0x7fa10c15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10a7fc700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cd1a0 0x7fa10c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 -- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cd1a0 msgr2=0x7fa10c0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cd1a0 0x7fa10c0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 -- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10c0cfa70 msgr2=0x7fa10c15e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10c0cfa70 0x7fa10c15e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 -- 172.21.15.92:0/47842699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009550 con 0x7fa10c0b2070 2023-06-09T04:36:01.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10b7fe700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10c0cfa70 0x7fa10c15e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:01.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10a7fc700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cd1a0 0x7fa10c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:01.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa10affd700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c158b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa10c066650 tx=0x7fa100004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa0f3fff700 1 -- 172.21.15.92:0/47842699 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100004310 con 0x7fa10c0b2070 2023-06-09T04:36:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa0f3fff700 1 -- 172.21.15.92:0/47842699 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10001f070 con 0x7fa10c0b2070 2023-06-09T04:36:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa0f3fff700 1 -- 172.21.15.92:0/47842699 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1000036a0 con 0x7fa10c0b2070 2023-06-09T04:36:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c0ca930 con 0x7fa10c0b2070 2023-06-09T04:36:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.383+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c0ca1a0 con 0x7fa10c0b2070 2023-06-09T04:36:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.384+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c0c66f0 con 0x7fa10c0b2070 2023-06-09T04:36:01.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.384+0000 7fa0f3fff700 1 -- 172.21.15.92:0/47842699 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa100021030 con 0x7fa10c0b2070 2023-06-09T04:36:01.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.385+0000 7fa0f3fff700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0f806ace0 0x7fa0f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.385+0000 7fa0f3fff700 1 -- 172.21.15.92:0/47842699 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa10008a960 con 0x7fa10c0b2070 2023-06-09T04:36:01.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.385+0000 7fa10a7fc700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0f806ace0 0x7fa0f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.385+0000 7fa10a7fc700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0f806ace0 0x7fa0f806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa10c1e86b0 tx=0x7fa0fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.389+0000 7fa0f3fff700 1 -- 172.21.15.92:0/47842699 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa10c0c66f0 con 0x7fa10c0b2070 2023-06-09T04:36:01.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.578+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa10c0740e0 con 0x7fa10c0b2070 2023-06-09T04:36:01.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.579+0000 7fa0f3fff700 1 -- 172.21.15.92:0/47842699 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fa100055a60 con 0x7fa10c0b2070 2023-06-09T04:36:01.580 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:01.580 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":26,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:35:59.229550+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:35:15.100764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T04:35:16.382513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:36:01.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0f806ace0 msgr2=0x7fa0f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0f806ace0 0x7fa0f806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa10c1e86b0 tx=0x7fa0fc008040 comp rx=0 tx=0).stop 2023-06-09T04:36:01.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 msgr2=0x7fa10c158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c158b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa10c066650 tx=0x7fa100004750 comp rx=0 tx=0).stop 2023-06-09T04:36:01.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 shutdown_connections 2023-06-09T04:36:01.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa0f806ace0 0x7fa0f806d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa10c0cfa70 0x7fa10c15e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c0b2070 0x7fa10c158b60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 --2- 172.21.15.92:0/47842699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa10c0cd1a0 0x7fa10c0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 >> 172.21.15.92:0/47842699 conn(0x7fa10c065a50 msgr2=0x7fa10c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:01.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 shutdown_connections 2023-06-09T04:36:01.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.581+0000 7fa11185a700 1 -- 172.21.15.92:0/47842699 wait complete. 2023-06-09T04:36:01.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-09T04:36:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.714+0000 7f36f3b9e700 1 Processor -- start 2023-06-09T04:36:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.714+0000 7f36f3b9e700 1 -- start start 2023-06-09T04:36:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f3b9e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36ec0cf010 0x7f36ec0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f3b9e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f36ec0ac400 0x7f36ec06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f3b9e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec0ce730 con 0x7f36ec15c7d0 2023-06-09T04:36:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f3b9e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec06aaf0 con 0x7f36ec0cf010 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f3b9e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec15cba0 con 0x7f36ec0ac400 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f1139700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36ec0cf010 0x7f36ec0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f1139700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36ec0cf010 0x7f36ec0ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39220/0 (socket says 172.21.15.92:39220) 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f1139700 1 -- 172.21.15.92:0/4223007462 learned_addr learned my addr 172.21.15.92:0/4223007462 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.715+0000 7f36f213b700 1 --2- 172.21.15.92:0/4223007462 >> v2:172.21.15.123:3300/0 conn(0x7f36ec0ac400 0x7f36ec06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f1139700 1 -- 172.21.15.92:0/4223007462 >> v2:172.21.15.123:3300/0 conn(0x7f36ec0ac400 msgr2=0x7f36ec06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f1139700 1 --2- 172.21.15.92:0/4223007462 >> v2:172.21.15.123:3300/0 conn(0x7f36ec0ac400 0x7f36ec06e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f1139700 1 -- 172.21.15.92:0/4223007462 >> v1:172.21.15.123:6789/0 conn(0x7f36ec15c7d0 legacy=0x7f36ec0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f1139700 1 -- 172.21.15.92:0/4223007462 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec069ee0 con 0x7f36ec0cf010 2023-06-09T04:36:01.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f213b700 1 --2- 172.21.15.92:0/4223007462 >> v2:172.21.15.123:3300/0 conn(0x7f36ec0ac400 0x7f36ec06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:01.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f1139700 1 --2- 172.21.15.92:0/4223007462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0cf010 0x7f36ec0ce1f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f36ec072180 tx=0x7f36e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cf91b9efb69f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f0938700 1 -- 172.21.15.92:0/4223007462 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e800b870 con 0x7f36ec0cf010 2023-06-09T04:36:01.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f0938700 1 -- 172.21.15.92:0/4223007462 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e800b9d0 con 0x7f36ec0cf010 2023-06-09T04:36:01.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f0938700 1 -- 172.21.15.92:0/4223007462 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e800bd60 con 0x7f36ec0cf010 2023-06-09T04:36:01.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4223007462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0cf010 msgr2=0x7f36ec0ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4223007462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0cf010 0x7f36ec0ce1f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f36ec072180 tx=0x7f36e8009bd0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4223007462 shutdown_connections 2023-06-09T04:36:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4223007462 >> v2:172.21.15.123:3300/0 conn(0x7f36ec0ac400 0x7f36ec06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.716+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4223007462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0cf010 0x7f36ec0ce1f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.717+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4223007462 >> 172.21.15.92:0/4223007462 conn(0x7f36ec098c10 msgr2=0x7f36ec0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.717+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4223007462 shutdown_connections 2023-06-09T04:36:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.717+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4223007462 wait complete. 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 Processor -- start 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 -- start start 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0ac400 0x7f36ec159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36ec0cf010 0x7f36ec0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec15c7d0 0x7f36ec15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0cab00 con 0x7f36ec0ac400 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0c3880 con 0x7f36ec15c7d0 2023-06-09T04:36:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.718+0000 7f36f3b9e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0c3b60 con 0x7f36ec0cf010 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0ac400 0x7f36ec159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0ac400 0x7f36ec159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39224/0 (socket says 172.21.15.92:39224) 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 -- 172.21.15.92:0/4153487319 learned_addr learned my addr 172.21.15.92:0/4153487319 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f1139700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36ec0cf010 0x7f36ec0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f213b700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec15c7d0 0x7f36ec15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 -- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36ec0cf010 msgr2=0x7f36ec0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36ec0cf010 0x7f36ec0cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 -- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec15c7d0 msgr2=0x7f36ec15af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec15c7d0 0x7f36ec15af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 -- 172.21.15.92:0/4153487319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8009550 con 0x7f36ec0ac400 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f193a700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0ac400 0x7f36ec159af0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c4d0 tx=0x7f36dc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36e27fc700 1 -- 172.21.15.92:0/4153487319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0052d0 con 0x7f36ec0ac400 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36e27fc700 1 -- 172.21.15.92:0/4153487319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc005430 con 0x7f36ec0ac400 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36e27fc700 1 -- 172.21.15.92:0/4153487319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0038f0 con 0x7f36ec0ac400 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ec0c3e40 con 0x7f36ec0ac400 2023-06-09T04:36:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.719+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec0c4120 con 0x7f36ec0ac400 2023-06-09T04:36:01.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.720+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec0740e0 con 0x7f36ec0ac400 2023-06-09T04:36:01.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.720+0000 7f36e27fc700 1 -- 172.21.15.92:0/4153487319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36dc003a50 con 0x7f36ec0ac400 2023-06-09T04:36:01.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.721+0000 7f36e27fc700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f36d006ace0 0x7f36d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:01.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.721+0000 7f36e27fc700 1 -- 172.21.15.92:0/4153487319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f36dc088680 con 0x7f36ec0ac400 2023-06-09T04:36:01.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.721+0000 7f36f1139700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f36d006ace0 0x7f36d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:01.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.721+0000 7f36f1139700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f36d006ace0 0x7f36d006d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0cf400 tx=0x7f36e8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:01.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.725+0000 7f36e27fc700 1 -- 172.21.15.92:0/4153487319 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f36dc053780 con 0x7f36ec0ac400 2023-06-09T04:36:01.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:01.953+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f36ec0ce730 con 0x7f36ec0ac400 2023-06-09T04:36:02.240 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-06-09T04:36:02.240+0000 7f45d913f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-06-09T04:36:02.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.251+0000 7f36e27fc700 1 -- 172.21.15.92:0/4153487319 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f36ec0ce730 con 0x7f36ec0ac400 2023-06-09T04:36:02.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f36d006ace0 msgr2=0x7f36d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:02.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f36d006ace0 0x7f36d006d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0cf400 tx=0x7f36e8008030 comp rx=0 tx=0).stop 2023-06-09T04:36:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0ac400 msgr2=0x7f36ec159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0ac400 0x7f36ec159af0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c4d0 tx=0x7f36dc00c7e0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 shutdown_connections 2023-06-09T04:36:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f36d006ace0 0x7f36d006d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec15c7d0 0x7f36ec15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36ec0ac400 0x7f36ec159af0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 --2- 172.21.15.92:0/4153487319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36ec0cf010 0x7f36ec0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 >> 172.21.15.92:0/4153487319 conn(0x7f36ec098c10 msgr2=0x7f36ec0af320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:02.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 shutdown_connections 2023-06-09T04:36:02.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.253+0000 7f36f3b9e700 1 -- 172.21.15.92:0/4153487319 wait complete. 2023-06-09T04:36:02.255 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-09T04:36:02.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-09T04:36:02.288 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-06-09T04:36:02.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.388+0000 7fdd47829700 1 Processor -- start 2023-06-09T04:36:02.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.388+0000 7fdd47829700 1 -- start start 2023-06-09T04:36:02.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd47829700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd400d03b0 0x7fdd400cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:02.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd47829700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd400cd8c0 0x7fdd400ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:02.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd47829700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd400cb700 con 0x7fdd400b60d0 2023-06-09T04:36:02.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd47829700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd400cdd00 con 0x7fdd400d03b0 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd47829700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd400b64a0 con 0x7fdd400cd8c0 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd44dc4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd400d03b0 0x7fdd400cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd455c5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdd400b60d0 0x7fdd4015e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37564/0 (socket says 172.21.15.92:37564) 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd455c5700 1 -- 172.21.15.92:0/3729568838 learned_addr learned my addr 172.21.15.92:0/3729568838 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd44dc4700 1 -- 172.21.15.92:0/3729568838 >> v2:172.21.15.123:3300/0 conn(0x7fdd400cd8c0 msgr2=0x7fdd400ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.389+0000 7fdd45dc6700 1 --2- 172.21.15.92:0/3729568838 >> v2:172.21.15.123:3300/0 conn(0x7fdd400cd8c0 0x7fdd400ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd44dc4700 1 --2- 172.21.15.92:0/3729568838 >> v2:172.21.15.123:3300/0 conn(0x7fdd400cd8c0 0x7fdd400ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd37fff700 1 -- 172.21.15.92:0/3729568838 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753022077 0 0) 0x7fdd30001110 con 0x7fdd400b60d0 2023-06-09T04:36:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd44dc4700 1 -- 172.21.15.92:0/3729568838 >> v1:172.21.15.92:6789/0 conn(0x7fdd400b60d0 legacy=0x7fdd4015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd44dc4700 1 -- 172.21.15.92:0/3729568838 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400ccf50 con 0x7fdd400d03b0 2023-06-09T04:36:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd45dc6700 1 --2- 172.21.15.92:0/3729568838 >> v2:172.21.15.123:3300/0 conn(0x7fdd400cd8c0 0x7fdd400ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd44dc4700 1 --2- 172.21.15.92:0/3729568838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd400d03b0 0x7fdd400cd380 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdd400af900 tx=0x7fdd3c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47944b75937c823 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd37fff700 1 -- 172.21.15.92:0/3729568838 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c011830 con 0x7fdd400d03b0 2023-06-09T04:36:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd37fff700 1 -- 172.21.15.92:0/3729568838 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c011990 con 0x7fdd400d03b0 2023-06-09T04:36:02.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd37fff700 1 -- 172.21.15.92:0/3729568838 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c011d20 con 0x7fdd400d03b0 2023-06-09T04:36:02.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 -- 172.21.15.92:0/3729568838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd400d03b0 msgr2=0x7fdd400cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:02.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 --2- 172.21.15.92:0/3729568838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd400d03b0 0x7fdd400cd380 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdd400af900 tx=0x7fdd3c009b20 comp rx=0 tx=0).stop 2023-06-09T04:36:02.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 -- 172.21.15.92:0/3729568838 shutdown_connections 2023-06-09T04:36:02.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 --2- 172.21.15.92:0/3729568838 >> v2:172.21.15.123:3300/0 conn(0x7fdd400cd8c0 0x7fdd400ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 --2- 172.21.15.92:0/3729568838 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd400d03b0 0x7fdd400cd380 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 -- 172.21.15.92:0/3729568838 >> 172.21.15.92:0/3729568838 conn(0x7fdd4006e4d0 msgr2=0x7fdd4006e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:02.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 -- 172.21.15.92:0/3729568838 shutdown_connections 2023-06-09T04:36:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.390+0000 7fdd47829700 1 -- 172.21.15.92:0/3729568838 wait complete. 2023-06-09T04:36:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 Processor -- start 2023-06-09T04:36:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 -- start start 2023-06-09T04:36:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400b60d0 0x7fdd400aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd400cd8c0 0x7fdd4006aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4006b400 0x7fdd4006b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:02.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4015b5b0 con 0x7fdd4006b400 2023-06-09T04:36:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd400af510 con 0x7fdd400b60d0 2023-06-09T04:36:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd47829700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4015a930 con 0x7fdd400cd8c0 2023-06-09T04:36:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.392+0000 7fdd45dc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4006b400 0x7fdd4006b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4006b400 0x7fdd4006b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39242/0 (socket says 172.21.15.92:39242) 2023-06-09T04:36:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 -- 172.21.15.92:0/946402716 learned_addr learned my addr 172.21.15.92:0/946402716 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd455c5700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400b60d0 0x7fdd400aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:02.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 -- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd400cd8c0 msgr2=0x7fdd4006aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd44dc4700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd400cd8c0 0x7fdd4006aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd400cd8c0 0x7fdd4006aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 -- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400b60d0 msgr2=0x7fdd400aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400b60d0 0x7fdd400aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 -- 172.21.15.92:0/946402716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c0094a0 con 0x7fdd4006b400 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd45dc6700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4006b400 0x7fdd4006b8a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800cd50 tx=0x7fdd3800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd35ffb700 1 -- 172.21.15.92:0/946402716 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800f480 con 0x7fdd4006b400 2023-06-09T04:36:02.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd35ffb700 1 -- 172.21.15.92:0/946402716 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3800f600 con 0x7fdd4006b400 2023-06-09T04:36:02.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4015ac10 con 0x7fdd4006b400 2023-06-09T04:36:02.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd35ffb700 1 -- 172.21.15.92:0/946402716 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800f9b0 con 0x7fdd4006b400 2023-06-09T04:36:02.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.393+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4015b1d0 con 0x7fdd4006b400 2023-06-09T04:36:02.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.394+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4015b6f0 con 0x7fdd4006b400 2023-06-09T04:36:02.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.394+0000 7fdd35ffb700 1 -- 172.21.15.92:0/946402716 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd3800b040 con 0x7fdd4006b400 2023-06-09T04:36:02.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.395+0000 7fdd35ffb700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdd2406ad10 0x7fdd2406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:02.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.395+0000 7fdd35ffb700 1 -- 172.21.15.92:0/946402716 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fdd38089cf0 con 0x7fdd4006b400 2023-06-09T04:36:02.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.395+0000 7fdd455c5700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdd2406ad10 0x7fdd2406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:02.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.395+0000 7fdd455c5700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdd2406ad10 0x7fdd2406d1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd30000f60 tx=0x7fdd30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:02.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.399+0000 7fdd35ffb700 1 -- 172.21.15.92:0/946402716 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd4015b6f0 con 0x7fdd4006b400 2023-06-09T04:36:02.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:02.630+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd4015b6f0 con 0x7fdd4006b400 2023-06-09T04:36:03.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.257+0000 7fdd35ffb700 1 -- 172.21.15.92:0/946402716 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd4015b6f0 con 0x7fdd4006b400 2023-06-09T04:36:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdd2406ad10 msgr2=0x7fdd2406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdd2406ad10 0x7fdd2406d1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd30000f60 tx=0x7fdd30009040 comp rx=0 tx=0).stop 2023-06-09T04:36:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4006b400 msgr2=0x7fdd4006b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4006b400 0x7fdd4006b8a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800cd50 tx=0x7fdd3800a560 comp rx=0 tx=0).stop 2023-06-09T04:36:03.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 shutdown_connections 2023-06-09T04:36:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdd2406ad10 0x7fdd2406d1c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd4006b400 0x7fdd4006b8a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd400b60d0 0x7fdd400aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 --2- 172.21.15.92:0/946402716 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd400cd8c0 0x7fdd4006aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 >> 172.21.15.92:0/946402716 conn(0x7fdd4006e4d0 msgr2=0x7fdd400b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 shutdown_connections 2023-06-09T04:36:03.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.259+0000 7fdd47829700 1 -- 172.21.15.92:0/946402716 wait complete. 2023-06-09T04:36:03.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-09T04:36:03.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.396+0000 7f1100180700 1 Processor -- start 2023-06-09T04:36:03.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.396+0000 7f1100180700 1 -- start start 2023-06-09T04:36:03.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f1100180700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10f80cf440 0x7f10f815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:03.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f1100180700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f80cce60 0x7f10f806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:03.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f1100180700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f815d040 con 0x7f10f8069db0 2023-06-09T04:36:03.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f1100180700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f8160ab0 con 0x7f10f80cf440 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f1100180700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f815ca10 con 0x7f10f80cce60 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f10fd71b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10f80cf440 0x7f10f815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f10fd71b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f10f80cf440 0x7f10f815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39248/0 (socket says 172.21.15.92:39248) 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.397+0000 7f10fd71b700 1 -- 172.21.15.92:0/3313255872 learned_addr learned my addr 172.21.15.92:0/3313255872 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fe71d700 1 --2- 172.21.15.92:0/3313255872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f80cce60 0x7f10f806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fd71b700 1 -- 172.21.15.92:0/3313255872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f80cce60 msgr2=0x7f10f806ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fd71b700 1 --2- 172.21.15.92:0/3313255872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f80cce60 0x7f10f806ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fd71b700 1 -- 172.21.15.92:0/3313255872 >> v1:172.21.15.123:6789/0 conn(0x7f10f8069db0 legacy=0x7f10f806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fd71b700 1 -- 172.21.15.92:0/3313255872 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8160d50 con 0x7f10f80cf440 2023-06-09T04:36:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fe71d700 1 --2- 172.21.15.92:0/3313255872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f80cce60 0x7f10f806ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fd71b700 1 --2- 172.21.15.92:0/3313255872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cf440 0x7f10f815d4d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10f40098f0 tx=0x7f10f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec65646237cce2a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fcf1a700 1 -- 172.21.15.92:0/3313255872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4004230 con 0x7f10f80cf440 2023-06-09T04:36:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fcf1a700 1 -- 172.21.15.92:0/3313255872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4004390 con 0x7f10f80cf440 2023-06-09T04:36:03.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f10fcf1a700 1 -- 172.21.15.92:0/3313255872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f40036a0 con 0x7f10f80cf440 2023-06-09T04:36:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f1100180700 1 -- 172.21.15.92:0/3313255872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cf440 msgr2=0x7f10f815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:03.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f1100180700 1 --2- 172.21.15.92:0/3313255872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cf440 0x7f10f815d4d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10f40098f0 tx=0x7f10f4009c00 comp rx=0 tx=0).stop 2023-06-09T04:36:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f1100180700 1 -- 172.21.15.92:0/3313255872 shutdown_connections 2023-06-09T04:36:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f1100180700 1 --2- 172.21.15.92:0/3313255872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f80cce60 0x7f10f806ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f1100180700 1 --2- 172.21.15.92:0/3313255872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cf440 0x7f10f815d4d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.398+0000 7f1100180700 1 -- 172.21.15.92:0/3313255872 >> 172.21.15.92:0/3313255872 conn(0x7f10f80cc380 msgr2=0x7f10f80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.399+0000 7f1100180700 1 -- 172.21.15.92:0/3313255872 shutdown_connections 2023-06-09T04:36:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.399+0000 7f1100180700 1 -- 172.21.15.92:0/3313255872 wait complete. 2023-06-09T04:36:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 Processor -- start 2023-06-09T04:36:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 -- start start 2023-06-09T04:36:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8069db0 0x7f10f80b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cce60 0x7f10f814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:03.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f815e520 0x7f10f815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f815ef00 con 0x7f10f80cce60 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80b3760 con 0x7f10f8069db0 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.400+0000 7f1100180700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f814f3b0 con 0x7f10f815e520 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cce60 0x7f10f814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cce60 0x7f10f814ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39254/0 (socket says 172.21.15.92:39254) 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 -- 172.21.15.92:0/3028660194 learned_addr learned my addr 172.21.15.92:0/3028660194 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fdf1c700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8069db0 0x7f10f80b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:03.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fe71d700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f815e520 0x7f10f815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 -- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f815e520 msgr2=0x7f10f815e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f815e520 0x7f10f815e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 -- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8069db0 msgr2=0x7f10f80b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8069db0 0x7f10f80b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 -- 172.21.15.92:0/3028660194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4009550 con 0x7f10f80cce60 2023-06-09T04:36:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fe71d700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f815e520 0x7f10f815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:03.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10fd71b700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cce60 0x7f10f814ee70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009840 tx=0x7f10f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:03.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10ee7fc700 1 -- 172.21.15.92:0/3028660194 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4003d80 con 0x7f10f80cce60 2023-06-09T04:36:03.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10ee7fc700 1 -- 172.21.15.92:0/3028660194 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f401f070 con 0x7f10f80cce60 2023-06-09T04:36:03.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80b2e30 con 0x7f10f80cce60 2023-06-09T04:36:03.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f10ee7fc700 1 -- 172.21.15.92:0/3028660194 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f400b8a0 con 0x7f10f80cce60 2023-06-09T04:36:03.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.401+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f80b33f0 con 0x7f10f80cce60 2023-06-09T04:36:03.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.402+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f80b35b0 con 0x7f10f80cce60 2023-06-09T04:36:03.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.403+0000 7f10ee7fc700 1 -- 172.21.15.92:0/3028660194 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f10f4021030 con 0x7f10f80cce60 2023-06-09T04:36:03.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.403+0000 7f10ee7fc700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f10dc06ace0 0x7f10dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:03.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.403+0000 7f10ee7fc700 1 -- 172.21.15.92:0/3028660194 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f10f408aab0 con 0x7f10f80cce60 2023-06-09T04:36:03.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.403+0000 7f10fdf1c700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f10dc06ace0 0x7f10dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.403+0000 7f10fdf1c700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f10dc06ace0 0x7f10dc06d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10f80b0780 tx=0x7f10e8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:03.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.407+0000 7f10ee7fc700 1 -- 172.21.15.92:0/3028660194 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f10f4055c80 con 0x7f10f80cce60 2023-06-09T04:36:03.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:03.594+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10f80b35b0 con 0x7f10f80cce60 2023-06-09T04:36:04.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.261+0000 7f10ee7fc700 1 -- 172.21.15.92:0/3028660194 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f10f40592a0 con 0x7f10f80cce60 2023-06-09T04:36:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f10dc06ace0 msgr2=0x7f10dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:04.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f10dc06ace0 0x7f10dc06d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10f80b0780 tx=0x7f10e8007ec0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cce60 msgr2=0x7f10f814ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cce60 0x7f10f814ee70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009840 tx=0x7f10f4004750 comp rx=0 tx=0).stop 2023-06-09T04:36:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 shutdown_connections 2023-06-09T04:36:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f10dc06ace0 0x7f10dc06d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10f815e520 0x7f10f815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10f8069db0 0x7f10f80b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 --2- 172.21.15.92:0/3028660194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10f80cce60 0x7f10f814ee70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 >> 172.21.15.92:0/3028660194 conn(0x7f10f80cc380 msgr2=0x7f10f80cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 shutdown_connections 2023-06-09T04:36:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.264+0000 7f1100180700 1 -- 172.21.15.92:0/3028660194 wait complete. 2023-06-09T04:36:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-06-09T04:36:04.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-09T04:36:04.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.408+0000 7fb62bad1700 1 Processor -- start 2023-06-09T04:36:04.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62bad1700 1 -- start start 2023-06-09T04:36:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62bad1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6240cce60 0x7fb62415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62bad1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62415d040 con 0x7fb624069db0 2023-06-09T04:36:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62bad1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb624160ab0 con 0x7fb6240cf440 2023-06-09T04:36:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62bad1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62415ca10 con 0x7fb6240cce60 2023-06-09T04:36:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62a06e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb6240cce60 0x7fb62415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:04.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62986d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb624069db0 0x7fb62406a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37592/0 (socket says 172.21.15.92:37592) 2023-06-09T04:36:04.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.409+0000 7fb62986d700 1 -- 172.21.15.92:0/3377991823 learned_addr learned my addr 172.21.15.92:0/3377991823 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:04.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.410+0000 7fb62a06e700 1 -- 172.21.15.92:0/3377991823 >> v1:172.21.15.123:6789/0 conn(0x7fb6240cf440 legacy=0x7fb62406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:04.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.410+0000 7fb62a06e700 1 -- 172.21.15.92:0/3377991823 >> v1:172.21.15.92:6789/0 conn(0x7fb624069db0 legacy=0x7fb62406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:04.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.410+0000 7fb62a06e700 1 -- 172.21.15.92:0/3377991823 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624160d50 con 0x7fb6240cce60 2023-06-09T04:36:04.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.410+0000 7fb62a06e700 1 --2- 172.21.15.92:0/3377991823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cce60 0x7fb62415d4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009d90 tx=0x7fb61c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5b275a958014d8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:04.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.410+0000 7fb62886b700 1 -- 172.21.15.92:0/3377991823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e070 con 0x7fb6240cce60 2023-06-09T04:36:04.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.410+0000 7fb62886b700 1 -- 172.21.15.92:0/3377991823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00bb90 con 0x7fb6240cce60 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.410+0000 7fb62886b700 1 -- 172.21.15.92:0/3377991823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c003a20 con 0x7fb6240cce60 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.411+0000 7fb62bad1700 1 -- 172.21.15.92:0/3377991823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cce60 msgr2=0x7fb62415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.411+0000 7fb62bad1700 1 --2- 172.21.15.92:0/3377991823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cce60 0x7fb62415d4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009d90 tx=0x7fb61c00b330 comp rx=0 tx=0).stop 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.411+0000 7fb62bad1700 1 -- 172.21.15.92:0/3377991823 shutdown_connections 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.411+0000 7fb62bad1700 1 --2- 172.21.15.92:0/3377991823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cce60 0x7fb62415d4d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.411+0000 7fb62bad1700 1 -- 172.21.15.92:0/3377991823 >> 172.21.15.92:0/3377991823 conn(0x7fb6240cc380 msgr2=0x7fb6240cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.411+0000 7fb62bad1700 1 -- 172.21.15.92:0/3377991823 shutdown_connections 2023-06-09T04:36:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.411+0000 7fb62bad1700 1 -- 172.21.15.92:0/3377991823 wait complete. 2023-06-09T04:36:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.412+0000 7fb62bad1700 1 Processor -- start 2023-06-09T04:36:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.412+0000 7fb62bad1700 1 -- start start 2023-06-09T04:36:04.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb624069db0 0x7fb624155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:04.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6240cce60 0x7fb6240c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:04.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cf440 0x7fb62415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:04.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240c6b00 con 0x7fb6240cf440 2023-06-09T04:36:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240c6de0 con 0x7fb624069db0 2023-06-09T04:36:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240c70c0 con 0x7fb6240cce60 2023-06-09T04:36:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cf440 0x7fb62415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cf440 0x7fb62415f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39270/0 (socket says 172.21.15.92:39270) 2023-06-09T04:36:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 -- 172.21.15.92:0/1648583185 learned_addr learned my addr 172.21.15.92:0/1648583185 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62906c700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6240cce60 0x7fb6240c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62986d700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb624069db0 0x7fb624155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 -- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6240cce60 msgr2=0x7fb6240c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6240cce60 0x7fb6240c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 -- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb624069db0 msgr2=0x7fb624155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb624069db0 0x7fb624155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 -- 172.21.15.92:0/1648583185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c009a70 con 0x7fb6240cf440 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62a06e700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cf440 0x7fb62415f0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb624070ca0 tx=0x7fb61c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb61a7fc700 1 -- 172.21.15.92:0/1648583185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012630 con 0x7fb6240cf440 2023-06-09T04:36:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb61a7fc700 1 -- 172.21.15.92:0/1648583185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c0127b0 con 0x7fb6240cf440 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6240c73a0 con 0x7fb6240cf440 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb61a7fc700 1 -- 172.21.15.92:0/1648583185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012b60 con 0x7fb6240cf440 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.413+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6240c8dc0 con 0x7fb6240cf440 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.414+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62406be50 con 0x7fb6240cf440 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.415+0000 7fb61a7fc700 1 -- 172.21.15.92:0/1648583185 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb61c012cc0 con 0x7fb6240cf440 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.415+0000 7fb61a7fc700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb60806ace0 0x7fb60806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.415+0000 7fb61a7fc700 1 -- 172.21.15.92:0/1648583185 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fb61c08bdd0 con 0x7fb6240cf440 2023-06-09T04:36:04.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.415+0000 7fb62986d700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb60806ace0 0x7fb60806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:04.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.415+0000 7fb62986d700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb60806ace0 0x7fb60806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb614000f60 tx=0x7fb614009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:04.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.419+0000 7fb61a7fc700 1 -- 172.21.15.92:0/1648583185 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb61c057030 con 0x7fb6240cf440 2023-06-09T04:36:04.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:04.603+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6240c8fe0 con 0x7fb6240cf440 2023-06-09T04:36:05.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.265+0000 7fb61a7fc700 1 -- 172.21.15.92:0/1648583185 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb61c05a650 con 0x7fb6240cf440 2023-06-09T04:36:05.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb60806ace0 msgr2=0x7fb60806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:05.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb60806ace0 0x7fb60806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb614000f60 tx=0x7fb614009040 comp rx=0 tx=0).stop 2023-06-09T04:36:05.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cf440 msgr2=0x7fb62415f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:05.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cf440 0x7fb62415f0e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb624070ca0 tx=0x7fb61c004ad0 comp rx=0 tx=0).stop 2023-06-09T04:36:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 shutdown_connections 2023-06-09T04:36:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb60806ace0 0x7fb60806d190 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb614000f60 tx=0x7fb614009040 comp rx=0 tx=0).stop 2023-06-09T04:36:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6240cf440 0x7fb62415f0e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb624069db0 0x7fb624155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.270+0000 7fb62bad1700 1 --2- 172.21.15.92:0/1648583185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6240cce60 0x7fb6240c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.271+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 >> 172.21.15.92:0/1648583185 conn(0x7fb6240cc380 msgr2=0x7fb62415e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:05.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.271+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 shutdown_connections 2023-06-09T04:36:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.271+0000 7fb62bad1700 1 -- 172.21.15.92:0/1648583185 wait complete. 2023-06-09T04:36:05.274 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-06-09T04:36:05.287 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-09T04:36:05.287 INFO:tasks.ceph.mds.a:Stopping old one... 2023-06-09T04:36:05.287 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-09T04:36:05.288 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T04:36:05.288 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-09T04:36:05.288 INFO:tasks.ceph.mds.c:Stopping old one... 2023-06-09T04:36:05.288 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-09T04:36:05.288 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T04:36:05.289 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T04:36:05.289 INFO:tasks.ceph.mds.b:Stopping old one... 2023-06-09T04:36:05.289 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T04:36:05.289 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T04:36:05.290 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T04:36:05.290 INFO:tasks.ceph.mds.d:Stopping old one... 2023-06-09T04:36:05.290 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T04:36:05.290 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T04:36:05.298 INFO:tasks.ceph.mds.a:Stopped 2023-06-09T04:36:05.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-09T04:36:05.299 INFO:tasks.ceph.mds.c:Stopped 2023-06-09T04:36:05.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-09T04:36:05.301 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T04:36:05.301 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T04:36:05.302 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T04:36:05.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T04:36:05.341 INFO:tasks.ceph.mds.a:Started 2023-06-09T04:36:05.342 DEBUG:teuthology.parallel:result is None 2023-06-09T04:36:05.343 INFO:tasks.ceph.mds.c:Started 2023-06-09T04:36:05.343 DEBUG:teuthology.parallel:result is None 2023-06-09T04:36:05.344 INFO:tasks.ceph.mds.d:Started 2023-06-09T04:36:05.344 DEBUG:teuthology.parallel:result is None 2023-06-09T04:36:05.345 INFO:tasks.ceph.mds.b:Started 2023-06-09T04:36:05.345 DEBUG:teuthology.parallel:result is None 2023-06-09T04:36:05.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-06-09T04:36:05.420 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-06-09T04:36:05.420 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-06-09T04:36:05.431 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2023-06-09T04:36:05.431 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2023-06-09T04:36:05.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.479+0000 7f42e5b23700 1 Processor -- start 2023-06-09T04:36:05.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.479+0000 7f42e5b23700 1 -- start start 2023-06-09T04:36:05.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42e5b23700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42e015a380 0x7f42e00ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:05.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42e5b23700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00cdd10 con 0x7f42e00afef0 2023-06-09T04:36:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42e5b23700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e015a750 con 0x7f42e00af9a0 2023-06-09T04:36:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42e5b23700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e006eca0 con 0x7f42e015a380 2023-06-09T04:36:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42df7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f42e015a380 0x7f42e00ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42deffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42e00afef0 0x7f42e006f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37632/0 (socket says 172.21.15.92:37632) 2023-06-09T04:36:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42deffd700 1 -- 172.21.15.92:0/260567692 learned_addr learned my addr 172.21.15.92:0/260567692 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42df7fe700 1 -- 172.21.15.92:0/260567692 >> v1:172.21.15.123:6789/0 conn(0x7f42e00af9a0 legacy=0x7f42e006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:05.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42df7fe700 1 -- 172.21.15.92:0/260567692 >> v1:172.21.15.92:6789/0 conn(0x7f42e00afef0 legacy=0x7f42e006f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42df7fe700 1 -- 172.21.15.92:0/260567692 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e00ce270 con 0x7f42e015a380 2023-06-09T04:36:05.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42df7fe700 1 --2- 172.21.15.92:0/260567692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00ce540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009d90 tx=0x7f42d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a148cd10e021101 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.480+0000 7f42de7fc700 1 -- 172.21.15.92:0/260567692 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d800e070 con 0x7f42e015a380 2023-06-09T04:36:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42de7fc700 1 -- 172.21.15.92:0/260567692 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d800bbd0 con 0x7f42e015a380 2023-06-09T04:36:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42de7fc700 1 -- 172.21.15.92:0/260567692 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8012430 con 0x7f42e015a380 2023-06-09T04:36:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42e5b23700 1 -- 172.21.15.92:0/260567692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 msgr2=0x7f42e00ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42e5b23700 1 --2- 172.21.15.92:0/260567692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00ce540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009d90 tx=0x7f42d800b330 comp rx=0 tx=0).stop 2023-06-09T04:36:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42e5b23700 1 -- 172.21.15.92:0/260567692 shutdown_connections 2023-06-09T04:36:05.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42e5b23700 1 --2- 172.21.15.92:0/260567692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00ce540 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:05.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42e5b23700 1 -- 172.21.15.92:0/260567692 >> 172.21.15.92:0/260567692 conn(0x7f42e0089fe0 msgr2=0x7f42e00d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:05.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42e5b23700 1 -- 172.21.15.92:0/260567692 shutdown_connections 2023-06-09T04:36:05.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.481+0000 7f42e5b23700 1 -- 172.21.15.92:0/260567692 wait complete. 2023-06-09T04:36:05.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 Processor -- start 2023-06-09T04:36:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 -- start start 2023-06-09T04:36:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42e00af9a0 0x7f42e0143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e00afef0 0x7f42e00c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00c4010 con 0x7f42e015a380 2023-06-09T04:36:05.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01441f0 con 0x7f42e00afef0 2023-06-09T04:36:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.483+0000 7f42e5b23700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00c5470 con 0x7f42e00af9a0 2023-06-09T04:36:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39306/0 (socket says 172.21.15.92:39306) 2023-06-09T04:36:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 -- 172.21.15.92:0/2076538659 learned_addr learned my addr 172.21.15.92:0/2076538659 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42df7fe700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42e00af9a0 0x7f42e0143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42deffd700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e00afef0 0x7f42e00c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:05.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 -- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42e00af9a0 msgr2=0x7f42e0143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42e00af9a0 0x7f42e0143cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 -- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e00afef0 msgr2=0x7f42e00c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e00afef0 0x7f42e00c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 -- 172.21.15.92:0/2076538659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8009a70 con 0x7f42e015a380 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42df7fe700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42e00af9a0 0x7f42e0143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42dffff700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00b54a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f42d000c420 tx=0x7f42d000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42c7fff700 1 -- 172.21.15.92:0/2076538659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00052d0 con 0x7f42e015a380 2023-06-09T04:36:05.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42c7fff700 1 -- 172.21.15.92:0/2076538659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d0005430 con 0x7f42e015a380 2023-06-09T04:36:05.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e00c4860 con 0x7f42e015a380 2023-06-09T04:36:05.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42c7fff700 1 -- 172.21.15.92:0/2076538659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00038a0 con 0x7f42e015a380 2023-06-09T04:36:05.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.484+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e00c4e20 con 0x7f42e015a380 2023-06-09T04:36:05.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.485+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e00c5620 con 0x7f42e015a380 2023-06-09T04:36:05.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.486+0000 7f42c7fff700 1 -- 172.21.15.92:0/2076538659 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42d0003a00 con 0x7f42e015a380 2023-06-09T04:36:05.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.486+0000 7f42c7fff700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f42cc06ad10 0x7f42cc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:05.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.486+0000 7f42c7fff700 1 -- 172.21.15.92:0/2076538659 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f42d0088490 con 0x7f42e015a380 2023-06-09T04:36:05.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.486+0000 7f42df7fe700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f42cc06ad10 0x7f42cc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:05.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.487+0000 7f42df7fe700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f42cc06ad10 0x7f42cc06d1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42d8014040 tx=0x7f42d8011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:05.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.491+0000 7f42c7fff700 1 -- 172.21.15.92:0/2076538659 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f42e00c5620 con 0x7f42e015a380 2023-06-09T04:36:05.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:05.694+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f42e00cf820 con 0x7f42e015a380 2023-06-09T04:36:06.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.485+0000 7f42c7fff700 1 -- 172.21.15.92:0/2076538659 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f42d00537c0 con 0x7f42e015a380 2023-06-09T04:36:06.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f42cc06ad10 msgr2=0x7f42cc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f42cc06ad10 0x7f42cc06d1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42d8014040 tx=0x7f42d8011000 comp rx=0 tx=0).stop 2023-06-09T04:36:06.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 msgr2=0x7f42e00b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00b54a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f42d000c420 tx=0x7f42d000c730 comp rx=0 tx=0).stop 2023-06-09T04:36:06.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 shutdown_connections 2023-06-09T04:36:06.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f42cc06ad10 0x7f42cc06d1c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e015a380 0x7f42e00b54a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42e00af9a0 0x7f42e0143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 --2- 172.21.15.92:0/2076538659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42e00afef0 0x7f42e00c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 >> 172.21.15.92:0/2076538659 conn(0x7f42e0089fe0 msgr2=0x7f42e00b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:06.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 shutdown_connections 2023-06-09T04:36:06.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.488+0000 7f42e5b23700 1 -- 172.21.15.92:0/2076538659 wait complete. 2023-06-09T04:36:06.491 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-06-09T04:36:06.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-06-09T04:36:06.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.625+0000 7f194725a700 1 Processor -- start 2023-06-09T04:36:06.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.625+0000 7f194725a700 1 -- start start 2023-06-09T04:36:06.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.626+0000 7f194725a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19400b59d0 0x7f19400b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:06.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.626+0000 7f194725a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19400b5e40 con 0x7f194015c430 2023-06-09T04:36:06.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.626+0000 7f194725a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19400cd540 con 0x7f19400b4b40 2023-06-09T04:36:06.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.626+0000 7f194725a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19400b4fc0 con 0x7f19400b59d0 2023-06-09T04:36:06.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.626+0000 7f193ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19400b59d0 0x7f19400b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:06.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.626+0000 7f1944ff6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f194015c430 0x7f19400cedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37648/0 (socket says 172.21.15.92:37648) 2023-06-09T04:36:06.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.626+0000 7f1944ff6700 1 -- 172.21.15.92:0/1256052730 learned_addr learned my addr 172.21.15.92:0/1256052730 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:06.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193ffff700 1 -- 172.21.15.92:0/1256052730 >> v1:172.21.15.123:6789/0 conn(0x7f19400b4b40 legacy=0x7f19400d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193ffff700 1 -- 172.21.15.92:0/1256052730 >> v1:172.21.15.92:6789/0 conn(0x7f194015c430 legacy=0x7f19400cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193f7fe700 1 -- 172.21.15.92:0/1256052730 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568741373 0 0) 0x7f19400b5e40 con 0x7f194015c430 2023-06-09T04:36:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193ffff700 1 -- 172.21.15.92:0/1256052730 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19400d1200 con 0x7f19400b59d0 2023-06-09T04:36:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193ffff700 1 --2- 172.21.15.92:0/1256052730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19400b59d0 0x7f19400b4600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f192c009840 tx=0x7f192c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba58dd45d6ed9492 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193f7fe700 1 -- 172.21.15.92:0/1256052730 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c004230 con 0x7f19400b59d0 2023-06-09T04:36:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193f7fe700 1 -- 172.21.15.92:0/1256052730 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c004390 con 0x7f19400b59d0 2023-06-09T04:36:06.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.627+0000 7f193f7fe700 1 -- 172.21.15.92:0/1256052730 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0036a0 con 0x7f19400b59d0 2023-06-09T04:36:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.628+0000 7f194725a700 1 -- 172.21.15.92:0/1256052730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19400b59d0 msgr2=0x7f19400b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.628+0000 7f194725a700 1 --2- 172.21.15.92:0/1256052730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19400b59d0 0x7f19400b4600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f192c009840 tx=0x7f192c009b50 comp rx=0 tx=0).stop 2023-06-09T04:36:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.628+0000 7f194725a700 1 -- 172.21.15.92:0/1256052730 shutdown_connections 2023-06-09T04:36:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.628+0000 7f194725a700 1 --2- 172.21.15.92:0/1256052730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19400b59d0 0x7f19400b4600 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.628+0000 7f194725a700 1 -- 172.21.15.92:0/1256052730 >> 172.21.15.92:0/1256052730 conn(0x7f19400cf600 msgr2=0x7f19400cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.628+0000 7f194725a700 1 -- 172.21.15.92:0/1256052730 shutdown_connections 2023-06-09T04:36:06.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.628+0000 7f194725a700 1 -- 172.21.15.92:0/1256052730 wait complete. 2023-06-09T04:36:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.629+0000 7f194725a700 1 Processor -- start 2023-06-09T04:36:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.629+0000 7f194725a700 1 -- start start 2023-06-09T04:36:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f194725a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19400b4b40 0x7f19400c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f194725a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19400b59d0 0x7f19401e1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:06.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f194725a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f194015c430 0x7f19401e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:06.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f194725a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1940146610 con 0x7f194015c430 2023-06-09T04:36:06.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f194725a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19400cc570 con 0x7f19400b4b40 2023-06-09T04:36:06.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f194725a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1940145ce0 con 0x7f19400b59d0 2023-06-09T04:36:06.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f194015c430 0x7f19401e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:06.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f194015c430 0x7f19401e1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39326/0 (socket says 172.21.15.92:39326) 2023-06-09T04:36:06.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 -- 172.21.15.92:0/2601049706 learned_addr learned my addr 172.21.15.92:0/2601049706 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f1944ff6700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19400b4b40 0x7f19400c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f193ffff700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19400b59d0 0x7f19401e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 -- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19400b59d0 msgr2=0x7f19401e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19400b59d0 0x7f19401e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 -- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19400b4b40 msgr2=0x7f19400c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19400b4b40 0x7f19400c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 -- 172.21.15.92:0/2601049706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c0094a0 con 0x7f194015c430 2023-06-09T04:36:06.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f19457f7700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f194015c430 0x7f19401e1c60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f193800cc10 tx=0x7f193800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f193d7fa700 1 -- 172.21.15.92:0/2601049706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193800abb0 con 0x7f194015c430 2023-06-09T04:36:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f193d7fa700 1 -- 172.21.15.92:0/2601049706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193800ad10 con 0x7f194015c430 2023-06-09T04:36:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1940145fc0 con 0x7f194015c430 2023-06-09T04:36:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.630+0000 7f193d7fa700 1 -- 172.21.15.92:0/2601049706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938012660 con 0x7f194015c430 2023-06-09T04:36:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.631+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19400c8ec0 con 0x7f194015c430 2023-06-09T04:36:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.631+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19400740e0 con 0x7f194015c430 2023-06-09T04:36:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.632+0000 7f193d7fa700 1 -- 172.21.15.92:0/2601049706 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f19380127c0 con 0x7f194015c430 2023-06-09T04:36:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.632+0000 7f193d7fa700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f192406ace0 0x7f192406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.632+0000 7f193d7fa700 1 -- 172.21.15.92:0/2601049706 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f19380898d0 con 0x7f194015c430 2023-06-09T04:36:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.632+0000 7f1944ff6700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f192406ace0 0x7f192406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.633+0000 7f1944ff6700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f192406ace0 0x7f192406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1934000f60 tx=0x7f1934007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.637+0000 7f193d7fa700 1 -- 172.21.15.92:0/2601049706 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1938054d60 con 0x7f194015c430 2023-06-09T04:36:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.901+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f19400cdb40 con 0x7f194015c430 2023-06-09T04:36:06.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.901+0000 7f193d7fa700 1 -- 172.21.15.92:0/2601049706 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f1938016020 con 0x7f194015c430 2023-06-09T04:36:06.903 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:06.903 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-06-09T04:36:06.903 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-06-09T04:36:06.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.903 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-06-09T04:36:06.903 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4JkkRWHGhAANRHu29QVa0w4jNy4uoHuxQ==", 2023-06-09T04:36:06.904 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.904 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.904 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-06-09T04:36:06.904 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-06-09T04:36:06.904 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4Jkq/ObJhAAwcBPqWUytLIzHn5o7SxDFg==", 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-06-09T04:36:06.905 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4Jkipe+IRAAg2IxPNEVdY1SZWDCIvW7pg==", 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.906 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4Jkyqi8LRAA+ZzBtsM0fkmU23EvTDskBA==", 2023-06-09T04:36:06.907 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-06-09T04:36:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-06-09T04:36:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.909 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-06-09T04:36:06.909 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDOq4JkBgz9DhAAtsLR0VX+r0N4C8xns2mSUQ==", 2023-06-09T04:36:06.909 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.909 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.909 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.909 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.909 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDRq4JkqGeCDhAAo38JWWjdNexDgVe4IlyN6A==", 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.910 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.911 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-06-09T04:36:06.911 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDUq4JkC1+GDRAAFnmFCsWYqNeqNxVNcRxw+g==", 2023-06-09T04:36:06.911 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.911 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDXq4Jk85MhDRAA6v4qBxXw6YMskDJ/9JJJew==", 2023-06-09T04:36:06.912 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-06-09T04:36:06.913 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDdq4Jk/oi0FBAAbtRF0o8GPx+kc+MH4qvbTg==", 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.914 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDgq4JkBnGxFBAAniJ+fC2f163u0u5IlAk6tw==", 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-06-09T04:36:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDjq4JkOllOFBAAuxY5IpMVak8cpjCU1RGQKQ==", 2023-06-09T04:36:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDmq4JkmTMTFBAA/eQl2z5o21anTixCLtyZFQ==", 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-06-09T04:36:06.917 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4JkUoD5MRAAx3w8zlcMnuTeKsysCg+Oxg==", 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.918 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4JkPankOhAAQqfsTOny57jlvaUiBDn1Kg==", 2023-06-09T04:36:06.919 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.920 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.920 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-06-09T04:36:06.920 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-06-09T04:36:06.920 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-06-09T04:36:06.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4JkoBCjNxAAwrikYjaSgWFfBT2TsAafVQ==", 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-06-09T04:36:06.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDLq4Jkrif6BBAAdoy3vg6SwBtDJjziNCZxiw==", 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-06-09T04:36:06.922 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA==", 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.923 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-06-09T04:36:06.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-06-09T04:36:06.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-06-09T04:36:06.924 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.924 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mds", 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDsq4JkzsGuERAAsTPeGiIlEGKbaTD8r/rNCw==", 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mds" 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mgr", 2023-06-09T04:36:06.925 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDsq4JkZsquERAACJDmTROFRGQmSxcmSidkQA==", 2023-06-09T04:36:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mgr" 2023-06-09T04:36:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-osd", 2023-06-09T04:36:06.926 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDsq4JkwdOuERAAFAGLZg9OyuXCh4kkRNx67g==", 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-osd" 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd", 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDsq4JkQ9uuERAAC7+im3iFUOqt5SkRbGmIxA==", 2023-06-09T04:36:06.927 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd" 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDsq4JkcOKuERAAlJ1/+0Cbvbee9Xr0tApSUw==", 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.928 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-06-09T04:36:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rgw", 2023-06-09T04:36:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDsq4Jk3emuERAAofm9WVuAv4Ch2C9R3ijIZQ==", 2023-06-09T04:36:06.929 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rgw" 2023-06-09T04:36:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.930 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4JkAq4gFxAAumlBS5Z3WMHfT6PFQ1VgxA==", 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:06.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-06-09T04:36:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQDKq4JkVsx/ExAA2ic1kHrajB36QXO1QRmwqg==", 2023-06-09T04:36:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-06-09T04:36:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-06-09T04:36:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-06-09T04:36:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-06-09T04:36:06.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:06.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-06-09T04:36:06.933 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-06-09T04:36:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f192406ace0 msgr2=0x7f192406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f192406ace0 0x7f192406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1934000f60 tx=0x7f1934007c90 comp rx=0 tx=0).stop 2023-06-09T04:36:06.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f194015c430 msgr2=0x7f19401e1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f194015c430 0x7f19401e1c60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f193800cc10 tx=0x7f193800a060 comp rx=0 tx=0).stop 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 shutdown_connections 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f192406ace0 0x7f192406d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f194015c430 0x7f19401e1c60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19400b4b40 0x7f19400c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 --2- 172.21.15.92:0/2601049706 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19400b59d0 0x7f19401e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 >> 172.21.15.92:0/2601049706 conn(0x7f19400cf600 msgr2=0x7f19400d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:06.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 shutdown_connections 2023-06-09T04:36:06.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:06.903+0000 7f194725a700 1 -- 172.21.15.92:0/2601049706 wait complete. 2023-06-09T04:36:06.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-06-09T04:36:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.055+0000 7f23a2842700 1 Processor -- start 2023-06-09T04:36:07.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.055+0000 7f23a2842700 1 -- start start 2023-06-09T04:36:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a2842700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0b1360 0x7f239c0b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a2842700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f239c0ae630 0x7f239c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a2842700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f239c0ac400 0x7f239c0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a2842700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239c0ae270 con 0x7f239c0ac400 2023-06-09T04:36:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a2842700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239c0adde0 con 0x7f239c0ae630 2023-06-09T04:36:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a2842700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0af0f0 con 0x7f239c0b1360 2023-06-09T04:36:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f239c0ac400 0x7f239c0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f239c0ac400 0x7f239c0ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39334/0 (socket says 172.21.15.92:39334) 2023-06-09T04:36:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 -- 172.21.15.92:0/2379990571 learned_addr learned my addr 172.21.15.92:0/2379990571 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f239b7fe700 1 --2- 172.21.15.92:0/2379990571 >> v2:172.21.15.123:3300/0 conn(0x7f239c0ae630 0x7f239c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 -- 172.21.15.92:0/2379990571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0b1360 msgr2=0x7f239c0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f239bfff700 1 --2- 172.21.15.92:0/2379990571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0b1360 0x7f239c0b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 --2- 172.21.15.92:0/2379990571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0b1360 0x7f239c0b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 -- 172.21.15.92:0/2379990571 >> v2:172.21.15.123:3300/0 conn(0x7f239c0ae630 msgr2=0x7f239c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 --2- 172.21.15.92:0/2379990571 >> v2:172.21.15.123:3300/0 conn(0x7f239c0ae630 0x7f239c0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f23a0ddf700 1 -- 172.21.15.92:0/2379990571 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c0b1c70 con 0x7f239c0ac400 2023-06-09T04:36:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.056+0000 7f239b7fe700 1 --2- 172.21.15.92:0/2379990571 >> v2:172.21.15.123:3300/0 conn(0x7f239c0ae630 0x7f239c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:07.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a0ddf700 1 --2- 172.21.15.92:0/2379990571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ac400 0x7f239c0ac880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f239c071090 tx=0x7f239400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501930c30b3d3a20 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f239affd700 1 -- 172.21.15.92:0/2379990571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239400e070 con 0x7f239c0ac400 2023-06-09T04:36:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f239affd700 1 -- 172.21.15.92:0/2379990571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239400b9c0 con 0x7f239c0ac400 2023-06-09T04:36:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f239affd700 1 -- 172.21.15.92:0/2379990571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239400bd70 con 0x7f239c0ac400 2023-06-09T04:36:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 -- 172.21.15.92:0/2379990571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ac400 msgr2=0x7f239c0ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 --2- 172.21.15.92:0/2379990571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ac400 0x7f239c0ac880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f239c071090 tx=0x7f239400b330 comp rx=0 tx=0).stop 2023-06-09T04:36:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 -- 172.21.15.92:0/2379990571 shutdown_connections 2023-06-09T04:36:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 --2- 172.21.15.92:0/2379990571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ac400 0x7f239c0ac880 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 --2- 172.21.15.92:0/2379990571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0b1360 0x7f239c0b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 --2- 172.21.15.92:0/2379990571 >> v2:172.21.15.123:3300/0 conn(0x7f239c0ae630 0x7f239c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 -- 172.21.15.92:0/2379990571 >> 172.21.15.92:0/2379990571 conn(0x7f239c098c10 msgr2=0x7f239c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 -- 172.21.15.92:0/2379990571 shutdown_connections 2023-06-09T04:36:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.057+0000 7f23a2842700 1 -- 172.21.15.92:0/2379990571 wait complete. 2023-06-09T04:36:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.058+0000 7f23a2842700 1 Processor -- start 2023-06-09T04:36:07.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f23a2842700 1 -- start start 2023-06-09T04:36:07.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f23a2842700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0ac400 0x7f239c146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f23a2842700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ae630 0x7f239c0c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f23a2842700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f239c0b1360 0x7f239c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f23a2842700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0c4890 con 0x7f239c0ae630 2023-06-09T04:36:07.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f23a2842700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0c3c80 con 0x7f239c0b1360 2023-06-09T04:36:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f23a2842700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0c3f60 con 0x7f239c0ac400 2023-06-09T04:36:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f239b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ae630 0x7f239c0c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f239b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ae630 0x7f239c0c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39338/0 (socket says 172.21.15.92:39338) 2023-06-09T04:36:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.059+0000 7f239b7fe700 1 -- 172.21.15.92:0/2002225391 learned_addr learned my addr 172.21.15.92:0/2002225391 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:07.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f239bfff700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0ac400 0x7f239c146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f23a0ddf700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f239c0b1360 0x7f239c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f239b7fe700 1 -- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0ac400 msgr2=0x7f239c146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f239b7fe700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0ac400 0x7f239c146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f239b7fe700 1 -- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f239c0b1360 msgr2=0x7f239c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f239b7fe700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f239c0b1360 0x7f239c0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f239b7fe700 1 -- 172.21.15.92:0/2002225391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2394009b30 con 0x7f239c0ae630 2023-06-09T04:36:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f23a0ddf700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f239c0b1360 0x7f239c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f239b7fe700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ae630 0x7f239c0c3380 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c740 tx=0x7f238c00ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f2398ff9700 1 -- 172.21.15.92:0/2002225391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c011800 con 0x7f239c0ae630 2023-06-09T04:36:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f2398ff9700 1 -- 172.21.15.92:0/2002225391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c011960 con 0x7f239c0ae630 2023-06-09T04:36:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c0c4240 con 0x7f239c0ae630 2023-06-09T04:36:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f2398ff9700 1 -- 172.21.15.92:0/2002225391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c011cd0 con 0x7f239c0ae630 2023-06-09T04:36:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.060+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c1471c0 con 0x7f239c0ae630 2023-06-09T04:36:07.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.061+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c0740e0 con 0x7f239c0ae630 2023-06-09T04:36:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.061+0000 7f2398ff9700 1 -- 172.21.15.92:0/2002225391 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f238c021350 con 0x7f239c0ae630 2023-06-09T04:36:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.062+0000 7f2398ff9700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f238806ace0 0x7f238806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.062+0000 7f2398ff9700 1 -- 172.21.15.92:0/2002225391 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f238c088fd0 con 0x7f239c0ae630 2023-06-09T04:36:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.062+0000 7f239bfff700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f238806ace0 0x7f238806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.062+0000 7f239bfff700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f238806ace0 0x7f238806d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f239000aaa0 tx=0x7f2390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.066+0000 7f2398ff9700 1 -- 172.21.15.92:0/2002225391 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f238c057e60 con 0x7f239c0ae630 2023-06-09T04:36:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.334+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f239c158400 con 0x7f239c0ae630 2023-06-09T04:36:07.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.339+0000 7f2398ff9700 1 -- 172.21.15.92:0/2002225391 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f238c0579f0 con 0x7f239c0ae630 2023-06-09T04:36:07.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f238806ace0 msgr2=0x7f238806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f238806ace0 0x7f238806d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f239000aaa0 tx=0x7f2390008040 comp rx=0 tx=0).stop 2023-06-09T04:36:07.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ae630 msgr2=0x7f239c0c3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ae630 0x7f239c0c3380 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c740 tx=0x7f238c00ca50 comp rx=0 tx=0).stop 2023-06-09T04:36:07.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 shutdown_connections 2023-06-09T04:36:07.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f238806ace0 0x7f238806d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f239c0b1360 0x7f239c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f239c0ac400 0x7f239c146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 --2- 172.21.15.92:0/2002225391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f239c0ae630 0x7f239c0c3380 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 >> 172.21.15.92:0/2002225391 conn(0x7f239c098c10 msgr2=0x7f239c0b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:07.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 shutdown_connections 2023-06-09T04:36:07.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.342+0000 7f23a2842700 1 -- 172.21.15.92:0/2002225391 wait complete. 2023-06-09T04:36:07.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-06-09T04:36:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.476+0000 7f0091319700 1 Processor -- start 2023-06-09T04:36:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.476+0000 7f0091319700 1 -- start start 2023-06-09T04:36:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.476+0000 7f0091319700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f008c0ced00 0x7f008c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.476+0000 7f0091319700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f008c0cfa70 0x7f008c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.476+0000 7f0091319700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008c0d17c0 con 0x7f008c0b2070 2023-06-09T04:36:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0091319700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c0cf150 con 0x7f008c0cfa70 2023-06-09T04:36:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0091319700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0cfee0 con 0x7f008c0ced00 2023-06-09T04:36:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f008c0b2070 0x7f008c0cd1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37676/0 (socket says 172.21.15.92:37676) 2023-06-09T04:36:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008affd700 1 -- 172.21.15.92:0/3760020087 learned_addr learned my addr 172.21.15.92:0/3760020087 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:07.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008a7fc700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f008c0ced00 0x7f008c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008b7fe700 1 --2- 172.21.15.92:0/3760020087 >> v2:172.21.15.123:3300/0 conn(0x7f008c0cfa70 0x7f008c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503129671 0 0) 0x7f007c001110 con 0x7f008c0b2070 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00740035b0 con 0x7f008c0b2070 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007618257 0 0) 0x7f00740035b0 con 0x7f008c0b2070 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f008c0ced00 msgr2=0x7f008c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f008c0ced00 0x7f008c0cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 >> v2:172.21.15.123:3300/0 conn(0x7f008c0cfa70 msgr2=0x7f008c0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 --2- 172.21.15.92:0/3760020087 >> v2:172.21.15.123:3300/0 conn(0x7f008c0cfa70 0x7f008c0b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c0d0720 con 0x7f008c0b2070 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f007c001800 con 0x7f008c0b2070 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008b7fe700 1 --2- 172.21.15.92:0/3760020087 >> v2:172.21.15.123:3300/0 conn(0x7f008c0cfa70 0x7f008c0b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 >> v1:172.21.15.92:6789/0 conn(0x7f008c0b2070 legacy=0x7f008c0cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0074003a00 0x7f0074003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00740043e0 0x7f00740048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0074004f20 0x7f007400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0cf150 con 0x7f00740043e0 2023-06-09T04:36:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0cfee0 con 0x7f0074004f20 2023-06-09T04:36:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007400bf50 con 0x7f0074003a00 2023-06-09T04:36:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008a7fc700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f008c0ced00 0x7f008c0cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:36:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008a7fc700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00740043e0 0x7f00740048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008affd700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0074003a00 0x7f0074003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.477+0000 7f008b7fe700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0074004f20 0x7f007400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f008a7fc700 1 -- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0074003a00 msgr2=0x7f0074003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f008a7fc700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0074003a00 0x7f0074003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f008a7fc700 1 -- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0074004f20 msgr2=0x7f007400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f008a7fc700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0074004f20 0x7f007400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f008a7fc700 1 -- 172.21.15.92:0/3760020087 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740035b0 con 0x7f00740043e0 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f008a7fc700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00740043e0 0x7f00740048b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f008c0b2440 tx=0x7f0074005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d3f3efdd1909c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080004030 con 0x7f00740043e0 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080004190 con 0x7f00740043e0 2023-06-09T04:36:07.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f0089ffb700 1 -- 172.21.15.92:0/3760020087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080004520 con 0x7f00740043e0 2023-06-09T04:36:07.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f0091319700 1 -- 172.21.15.92:0/3760020087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00740043e0 msgr2=0x7f00740048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f0091319700 1 --2- 172.21.15.92:0/3760020087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00740043e0 0x7f00740048b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f008c0b2440 tx=0x7f0074005880 comp rx=0 tx=0).stop 2023-06-09T04:36:07.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.478+0000 7f008b7fe700 1 -- 172.21.15.92:0/3760020087 reap_dead start 2023-06-09T04:36:07.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.479+0000 7f0091319700 1 -- 172.21.15.92:0/3760020087 shutdown_connections 2023-06-09T04:36:07.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.479+0000 7f0091319700 1 -- 172.21.15.92:0/3760020087 >> 172.21.15.92:0/3760020087 conn(0x7f008c065a50 msgr2=0x7f008c065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:07.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.479+0000 7f0091319700 1 -- 172.21.15.92:0/3760020087 shutdown_connections 2023-06-09T04:36:07.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.479+0000 7f0091319700 1 -- 172.21.15.92:0/3760020087 wait complete. 2023-06-09T04:36:07.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.480+0000 7f0091319700 1 Processor -- start 2023-06-09T04:36:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.480+0000 7f0091319700 1 -- start start 2023-06-09T04:36:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00740043e0 0x7f008c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f008c1f5c00 0x7f008c1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f008c0c6f20 0x7f008c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0c6a90 con 0x7f008c1f5c00 2023-06-09T04:36:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1f6450 con 0x7f008c0c6f20 2023-06-09T04:36:07.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1f6730 con 0x7f00740043e0 2023-06-09T04:36:07.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f008c1f5c00 0x7f008c1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f008c1f5c00 0x7f008c1520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39358/0 (socket says 172.21.15.92:39358) 2023-06-09T04:36:07.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 -- 172.21.15.92:0/4264415746 learned_addr learned my addr 172.21.15.92:0/4264415746 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:07.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008affd700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00740043e0 0x7f008c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008b7fe700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f008c0c6f20 0x7f008c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 -- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00740043e0 msgr2=0x7f008c0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00740043e0 0x7f008c0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 -- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f008c0c6f20 msgr2=0x7f008c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f008c0c6f20 0x7f008c0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 -- 172.21.15.92:0/4264415746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c1f6a10 con 0x7f008c1f5c00 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f008a7fc700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f008c1f5c00 0x7f008c1520e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0080015000 tx=0x7f00800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f00737fe700 1 -- 172.21.15.92:0/4264415746 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080004030 con 0x7f008c1f5c00 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f00737fe700 1 -- 172.21.15.92:0/4264415746 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008000e890 con 0x7f008c1f5c00 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f00737fe700 1 -- 172.21.15.92:0/4264415746 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000ec40 con 0x7f008c1f5c00 2023-06-09T04:36:07.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c0c6720 con 0x7f008c1f5c00 2023-06-09T04:36:07.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.481+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c1f6070 con 0x7f008c1f5c00 2023-06-09T04:36:07.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.482+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c0c6c00 con 0x7f008c1f5c00 2023-06-09T04:36:07.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.483+0000 7f00737fe700 1 -- 172.21.15.92:0/4264415746 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0080013470 con 0x7f008c1f5c00 2023-06-09T04:36:07.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.483+0000 7f00737fe700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f007806ace0 0x7f007806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.483+0000 7f00737fe700 1 -- 172.21.15.92:0/4264415746 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f008008a380 con 0x7f008c1f5c00 2023-06-09T04:36:07.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.483+0000 7f008affd700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f007806ace0 0x7f007806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.484+0000 7f008affd700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f007806ace0 0x7f007806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f007c0012b0 tx=0x7f007c007b00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.487+0000 7f00737fe700 1 -- 172.21.15.92:0/4264415746 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0080055810 con 0x7f008c1f5c00 2023-06-09T04:36:07.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.753+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f008c0740e0 con 0x7f008c1f5c00 2023-06-09T04:36:07.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.758+0000 7f00737fe700 1 -- 172.21.15.92:0/4264415746 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0080058e30 con 0x7f008c1f5c00 2023-06-09T04:36:07.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f007806ace0 msgr2=0x7f007806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f007806ace0 0x7f007806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f007c0012b0 tx=0x7f007c007b00 comp rx=0 tx=0).stop 2023-06-09T04:36:07.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f008c1f5c00 msgr2=0x7f008c1520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f008c1f5c00 0x7f008c1520e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0080015000 tx=0x7f00800047c0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 shutdown_connections 2023-06-09T04:36:07.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f007806ace0 0x7f007806d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f008c0c6f20 0x7f008c0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f00740043e0 0x7f008c0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 --2- 172.21.15.92:0/4264415746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f008c1f5c00 0x7f008c1520e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 >> 172.21.15.92:0/4264415746 conn(0x7f008c065a50 msgr2=0x7f008c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:07.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.760+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 shutdown_connections 2023-06-09T04:36:07.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.761+0000 7f0091319700 1 -- 172.21.15.92:0/4264415746 wait complete. 2023-06-09T04:36:07.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-06-09T04:36:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.894+0000 7f57b8185700 1 Processor -- start 2023-06-09T04:36:07.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.894+0000 7f57b8185700 1 -- start start 2023-06-09T04:36:07.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b8185700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57b015a8d0 0x7f57b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b8185700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b00ac400 con 0x7f57b006a2f0 2023-06-09T04:36:07.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b8185700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b00cd7e0 con 0x7f57b015a500 2023-06-09T04:36:07.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b8185700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b006a790 con 0x7f57b015a8d0 2023-06-09T04:36:07.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b5720700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f57b015a8d0 0x7f57b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b6722700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f57b006a2f0 0x7f57b00ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37700/0 (socket says 172.21.15.92:37700) 2023-06-09T04:36:07.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b6722700 1 -- 172.21.15.92:0/2257993445 learned_addr learned my addr 172.21.15.92:0/2257993445 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:07.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.895+0000 7f57b5720700 1 -- 172.21.15.92:0/2257993445 >> v1:172.21.15.123:6789/0 conn(0x7f57b015a500 legacy=0x7f57b00cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:07.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b5720700 1 -- 172.21.15.92:0/2257993445 >> v1:172.21.15.92:6789/0 conn(0x7f57b006a2f0 legacy=0x7f57b00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b4f1f700 1 -- 172.21.15.92:0/2257993445 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993442698 0 0) 0x7f57b00ac400 con 0x7f57b006a2f0 2023-06-09T04:36:07.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b5720700 1 -- 172.21.15.92:0/2257993445 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00cdda0 con 0x7f57b015a8d0 2023-06-09T04:36:07.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b5720700 1 --2- 172.21.15.92:0/2257993445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a8d0 0x7f57b0069db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009840 tx=0x7f57ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4512bbaa0d738a97 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b4f1f700 1 -- 172.21.15.92:0/2257993445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac011830 con 0x7f57b015a8d0 2023-06-09T04:36:07.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b4f1f700 1 -- 172.21.15.92:0/2257993445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac011990 con 0x7f57b015a8d0 2023-06-09T04:36:07.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b4f1f700 1 -- 172.21.15.92:0/2257993445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac011d20 con 0x7f57b015a8d0 2023-06-09T04:36:07.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b8185700 1 -- 172.21.15.92:0/2257993445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a8d0 msgr2=0x7f57b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b8185700 1 --2- 172.21.15.92:0/2257993445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a8d0 0x7f57b0069db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009840 tx=0x7f57ac009b50 comp rx=0 tx=0).stop 2023-06-09T04:36:07.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b8185700 1 -- 172.21.15.92:0/2257993445 shutdown_connections 2023-06-09T04:36:07.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b8185700 1 --2- 172.21.15.92:0/2257993445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a8d0 0x7f57b0069db0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b8185700 1 -- 172.21.15.92:0/2257993445 >> 172.21.15.92:0/2257993445 conn(0x7f57b0098c10 msgr2=0x7f57b00cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b8185700 1 -- 172.21.15.92:0/2257993445 shutdown_connections 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.896+0000 7f57b8185700 1 -- 172.21.15.92:0/2257993445 wait complete. 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 Processor -- start 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 -- start start 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57b006a2f0 0x7f57b015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a500 0x7f57b01f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b015a8d0 0x7f57b00c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b015cef0 con 0x7f57b015a500 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00c73e0 con 0x7f57b015a8d0 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b8185700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00c76c0 con 0x7f57b006a2f0 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b5720700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a500 0x7f57b01f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b5720700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a500 0x7f57b01f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39372/0 (socket says 172.21.15.92:39372) 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.898+0000 7f57b5720700 1 -- 172.21.15.92:0/4176906226 learned_addr learned my addr 172.21.15.92:0/4176906226 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b6722700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b015a8d0 0x7f57b00c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b5720700 1 -- 172.21.15.92:0/4176906226 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57b006a2f0 msgr2=0x7f57b015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b5720700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57b006a2f0 0x7f57b015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b5720700 1 -- 172.21.15.92:0/4176906226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b015a8d0 msgr2=0x7f57b00c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b5720700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b015a8d0 0x7f57b00c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b5720700 1 -- 172.21.15.92:0/4176906226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0094a0 con 0x7f57b015a500 2023-06-09T04:36:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b5720700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a500 0x7f57b01f0d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f57ac014040 tx=0x7f57ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57a67fc700 1 -- 172.21.15.92:0/4176906226 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac01a900 con 0x7f57b015a500 2023-06-09T04:36:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57a67fc700 1 -- 172.21.15.92:0/4176906226 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac01aa60 con 0x7f57b015a500 2023-06-09T04:36:07.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b00c79a0 con 0x7f57b015a500 2023-06-09T04:36:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57a67fc700 1 -- 172.21.15.92:0/4176906226 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac01add0 con 0x7f57b015a500 2023-06-09T04:36:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.899+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b015afb0 con 0x7f57b015a500 2023-06-09T04:36:07.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.900+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b00740e0 con 0x7f57b015a500 2023-06-09T04:36:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.900+0000 7f57a67fc700 1 -- 172.21.15.92:0/4176906226 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57ac022070 con 0x7f57b015a500 2023-06-09T04:36:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.901+0000 7f57a67fc700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f579406ace0 0x7f579406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.901+0000 7f57a67fc700 1 -- 172.21.15.92:0/4176906226 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f57ac08a3e0 con 0x7f57b015a500 2023-06-09T04:36:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.901+0000 7f57b5f21700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f579406ace0 0x7f579406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.901+0000 7f57b5f21700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f579406ace0 0x7f579406d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57a0000fb0 tx=0x7f57a0007a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:07.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:07.905+0000 7f57a67fc700 1 -- 172.21.15.92:0/4176906226 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57ac055870 con 0x7f57b015a500 2023-06-09T04:36:08.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.170+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f57b01e2fa0 con 0x7f57b015a500 2023-06-09T04:36:08.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.174+0000 7f57a67fc700 1 -- 172.21.15.92:0/4176906226 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f57ac006cc0 con 0x7f57b015a500 2023-06-09T04:36:08.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f579406ace0 msgr2=0x7f579406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f579406ace0 0x7f579406d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57a0000fb0 tx=0x7f57a0007a30 comp rx=0 tx=0).stop 2023-06-09T04:36:08.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a500 msgr2=0x7f57b01f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a500 0x7f57b01f0d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f57ac014040 tx=0x7f57ac004750 comp rx=0 tx=0).stop 2023-06-09T04:36:08.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 shutdown_connections 2023-06-09T04:36:08.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f579406ace0 0x7f579406d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57b015a8d0 0x7f57b00c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f57b006a2f0 0x7f57b015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 --2- 172.21.15.92:0/4176906226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57b015a500 0x7f57b01f0d30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 >> 172.21.15.92:0/4176906226 conn(0x7f57b0098c10 msgr2=0x7f57b00ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:08.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 shutdown_connections 2023-06-09T04:36:08.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.176+0000 7f57b8185700 1 -- 172.21.15.92:0/4176906226 wait complete. 2023-06-09T04:36:08.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-06-09T04:36:08.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.310+0000 7fd091f9a700 1 Processor -- start 2023-06-09T04:36:08.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.310+0000 7fd091f9a700 1 -- start start 2023-06-09T04:36:08.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd091f9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd08c15c7d0 0x7fd08c0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd091f9a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c0ce730 con 0x7fd08c0ac400 2023-06-09T04:36:08.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd091f9a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c06aaf0 con 0x7fd08c0cf010 2023-06-09T04:36:08.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd091f9a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c15cba0 con 0x7fd08c15c7d0 2023-06-09T04:36:08.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd08c15c7d0 0x7fd08c0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd08c0ac400 0x7fd08c0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37710/0 (socket says 172.21.15.92:37710) 2023-06-09T04:36:08.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08affd700 1 -- 172.21.15.92:0/215412480 learned_addr learned my addr 172.21.15.92:0/215412480 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:08.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08b7fe700 1 -- 172.21.15.92:0/215412480 >> v1:172.21.15.123:6789/0 conn(0x7fd08c0cf010 legacy=0x7fd08c06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08b7fe700 1 -- 172.21.15.92:0/215412480 >> v1:172.21.15.92:6789/0 conn(0x7fd08c0ac400 legacy=0x7fd08c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08b7fe700 1 -- 172.21.15.92:0/215412480 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c069ee0 con 0x7fd08c15c7d0 2023-06-09T04:36:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08b7fe700 1 --2- 172.21.15.92:0/215412480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c15c7d0 0x7fd08c0ce1f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009820 tx=0x7fd07c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb52ef8776c21da server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08a7fc700 1 -- 172.21.15.92:0/215412480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c005e30 con 0x7fd08c15c7d0 2023-06-09T04:36:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08a7fc700 1 -- 172.21.15.92:0/215412480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c006030 con 0x7fd08c15c7d0 2023-06-09T04:36:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.311+0000 7fd08a7fc700 1 -- 172.21.15.92:0/215412480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0063e0 con 0x7fd08c15c7d0 2023-06-09T04:36:08.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.312+0000 7fd091f9a700 1 -- 172.21.15.92:0/215412480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c15c7d0 msgr2=0x7fd08c0ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.312+0000 7fd091f9a700 1 --2- 172.21.15.92:0/215412480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c15c7d0 0x7fd08c0ce1f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009820 tx=0x7fd07c009b30 comp rx=0 tx=0).stop 2023-06-09T04:36:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.312+0000 7fd091f9a700 1 -- 172.21.15.92:0/215412480 shutdown_connections 2023-06-09T04:36:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.312+0000 7fd091f9a700 1 --2- 172.21.15.92:0/215412480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c15c7d0 0x7fd08c0ce1f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.312+0000 7fd091f9a700 1 -- 172.21.15.92:0/215412480 >> 172.21.15.92:0/215412480 conn(0x7fd08c098c10 msgr2=0x7fd08c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.312+0000 7fd091f9a700 1 -- 172.21.15.92:0/215412480 shutdown_connections 2023-06-09T04:36:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.312+0000 7fd091f9a700 1 -- 172.21.15.92:0/215412480 wait complete. 2023-06-09T04:36:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.313+0000 7fd091f9a700 1 Processor -- start 2023-06-09T04:36:08.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.313+0000 7fd091f9a700 1 -- start start 2023-06-09T04:36:08.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd091f9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c0ac400 0x7fd08c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd091f9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c0cf010 0x7fd08c14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd091f9a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c15c7d0 0x7fd08c0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd091f9a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b3dd0 con 0x7fd08c0ac400 2023-06-09T04:36:08.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd091f9a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b31c0 con 0x7fd08c0cf010 2023-06-09T04:36:08.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd091f9a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b34a0 con 0x7fd08c15c7d0 2023-06-09T04:36:08.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c0ac400 0x7fd08c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c0ac400 0x7fd08c15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39382/0 (socket says 172.21.15.92:39382) 2023-06-09T04:36:08.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 -- 172.21.15.92:0/3338871621 learned_addr learned my addr 172.21.15.92:0/3338871621 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:08.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08affd700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c0cf010 0x7fd08c14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 -- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c15c7d0 msgr2=0x7fd08c0c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08bfff700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c15c7d0 0x7fd08c0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c15c7d0 0x7fd08c0c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 -- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c0cf010 msgr2=0x7fd08c14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c0cf010 0x7fd08c14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08b7fe700 1 -- 172.21.15.92:0/3338871621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c009430 con 0x7fd08c0ac400 2023-06-09T04:36:08.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08affd700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c0cf010 0x7fd08c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:08.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.314+0000 7fd08bfff700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c15c7d0 0x7fd08c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:08.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.315+0000 7fd08b7fe700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c0ac400 0x7fd08c15ad10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd07c000c00 tx=0x7fd07c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:08.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.315+0000 7fd073fff700 1 -- 172.21.15.92:0/3338871621 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004240 con 0x7fd08c0ac400 2023-06-09T04:36:08.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.315+0000 7fd073fff700 1 -- 172.21.15.92:0/3338871621 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0043e0 con 0x7fd08c0ac400 2023-06-09T04:36:08.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.315+0000 7fd073fff700 1 -- 172.21.15.92:0/3338871621 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0036a0 con 0x7fd08c0ac400 2023-06-09T04:36:08.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.315+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c0b3780 con 0x7fd08c0ac400 2023-06-09T04:36:08.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.315+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c14e230 con 0x7fd08c0ac400 2023-06-09T04:36:08.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.316+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c0740e0 con 0x7fd08c0ac400 2023-06-09T04:36:08.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.316+0000 7fd073fff700 1 -- 172.21.15.92:0/3338871621 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd07c022070 con 0x7fd08c0ac400 2023-06-09T04:36:08.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.316+0000 7fd073fff700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd07806ace0 0x7fd07806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.317+0000 7fd073fff700 1 -- 172.21.15.92:0/3338871621 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd07c08aad0 con 0x7fd08c0ac400 2023-06-09T04:36:08.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.317+0000 7fd08affd700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd07806ace0 0x7fd07806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.317+0000 7fd08affd700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd07806ace0 0x7fd07806d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b30f0 tx=0x7fd080009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:08.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.321+0000 7fd073fff700 1 -- 172.21.15.92:0/3338871621 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd07c055f60 con 0x7fd08c0ac400 2023-06-09T04:36:08.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.585+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fd08c15b4c0 con 0x7fd08c0ac400 2023-06-09T04:36:08.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.589+0000 7fd073fff700 1 -- 172.21.15.92:0/3338871621 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd07c02a0a0 con 0x7fd08c0ac400 2023-06-09T04:36:08.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd07806ace0 msgr2=0x7fd07806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd07806ace0 0x7fd07806d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b30f0 tx=0x7fd080009040 comp rx=0 tx=0).stop 2023-06-09T04:36:08.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c0ac400 msgr2=0x7fd08c15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c0ac400 0x7fd08c15ad10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd07c000c00 tx=0x7fd07c004750 comp rx=0 tx=0).stop 2023-06-09T04:36:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 shutdown_connections 2023-06-09T04:36:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd07806ace0 0x7fd07806d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd08c15c7d0 0x7fd08c0c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd08c0ac400 0x7fd08c15ad10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 --2- 172.21.15.92:0/3338871621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08c0cf010 0x7fd08c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 >> 172.21.15.92:0/3338871621 conn(0x7fd08c098c10 msgr2=0x7fd08c0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 shutdown_connections 2023-06-09T04:36:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.591+0000 7fd091f9a700 1 -- 172.21.15.92:0/3338871621 wait complete. 2023-06-09T04:36:08.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-06-09T04:36:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.723+0000 7f72da67e700 1 Processor -- start 2023-06-09T04:36:08.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.723+0000 7f72da67e700 1 -- start start 2023-06-09T04:36:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72da67e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 0x7f72d406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72da67e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d406ef00 con 0x7f72d40cfe60 2023-06-09T04:36:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72da67e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d406f1a0 con 0x7f72d406b7e0 2023-06-09T04:36:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72da67e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d406f480 con 0x7f72d415c850 2023-06-09T04:36:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72d37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f72d40cfe60 0x7f72d406b0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37726/0 (socket says 172.21.15.92:37726) 2023-06-09T04:36:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72d37fe700 1 -- 172.21.15.92:0/2967793622 learned_addr learned my addr 172.21.15.92:0/2967793622 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246382433 0 0) 0x7f72d406ef00 con 0x7f72d40cfe60 2023-06-09T04:36:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72d3fff700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 0x7f72d406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.724+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72bc0035b0 con 0x7f72d40cfe60 2023-06-09T04:36:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 525917345 0 0) 0x7f72bc0035b0 con 0x7f72d40cfe60 2023-06-09T04:36:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 msgr2=0x7f72d406ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 0x7f72d406ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 >> v1:172.21.15.123:6789/0 conn(0x7f72d406b7e0 legacy=0x7f72d40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d406f760 con 0x7f72d40cfe60 2023-06-09T04:36:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f72c4001670 con 0x7f72d40cfe60 2023-06-09T04:36:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 >> v1:172.21.15.92:6789/0 conn(0x7f72d40cfe60 legacy=0x7f72d406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc003a30 0x7f72bc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72bc004850 0x7f72bc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 0x7f72bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d406f480 con 0x7f72bc003a30 2023-06-09T04:36:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc00bb50 con 0x7f72bc004850 2023-06-09T04:36:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc00bce0 con 0x7f72d415c850 2023-06-09T04:36:08.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d3fff700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc003a30 0x7f72bc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d37fe700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72bc004850 0x7f72bc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d8c1b700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 0x7f72bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d3fff700 1 -- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 msgr2=0x7f72bc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d3fff700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d415c850 0x7f72bc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d3fff700 1 -- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72bc004850 msgr2=0x7f72bc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d3fff700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72bc004850 0x7f72bc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d3fff700 1 -- 172.21.15.92:0/2967793622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0035b0 con 0x7f72bc003a30 2023-06-09T04:36:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d3fff700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc003a30 0x7f72bc003e40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009b60 tx=0x7f72bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab8fa93e1b59ba0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800e6c0 con 0x7f72bc003a30 2023-06-09T04:36:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c800e820 con 0x7f72bc003a30 2023-06-09T04:36:08.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.725+0000 7f72d2ffd700 1 -- 172.21.15.92:0/2967793622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800eb90 con 0x7f72bc003a30 2023-06-09T04:36:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.726+0000 7f72da67e700 1 -- 172.21.15.92:0/2967793622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc003a30 msgr2=0x7f72bc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.726+0000 7f72da67e700 1 --2- 172.21.15.92:0/2967793622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72bc003a30 0x7f72bc003e40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009b60 tx=0x7f72bc005880 comp rx=0 tx=0).stop 2023-06-09T04:36:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.726+0000 7f72d8c1b700 1 -- 172.21.15.92:0/2967793622 reap_dead start 2023-06-09T04:36:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.726+0000 7f72da67e700 1 -- 172.21.15.92:0/2967793622 shutdown_connections 2023-06-09T04:36:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.726+0000 7f72da67e700 1 -- 172.21.15.92:0/2967793622 >> 172.21.15.92:0/2967793622 conn(0x7f72d4089fe0 msgr2=0x7f72d40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.726+0000 7f72da67e700 1 -- 172.21.15.92:0/2967793622 shutdown_connections 2023-06-09T04:36:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.726+0000 7f72da67e700 1 -- 172.21.15.92:0/2967793622 wait complete. 2023-06-09T04:36:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.727+0000 7f72da67e700 1 Processor -- start 2023-06-09T04:36:08.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.727+0000 7f72da67e700 1 -- start start 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72da67e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d41574c0 0x7f72d40ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72da67e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4157b30 0x7f72d40c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72da67e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72d415eda0 0x7f72d415f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72da67e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d415f750 con 0x7f72d415eda0 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72da67e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40b0a90 con 0x7f72d4157b30 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72da67e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40c9010 con 0x7f72d41574c0 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72d8c1b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72d415eda0 0x7f72d415f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72d8c1b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72d415eda0 0x7f72d415f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39406/0 (socket says 172.21.15.92:39406) 2023-06-09T04:36:08.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72d8c1b700 1 -- 172.21.15.92:0/3014254083 learned_addr learned my addr 172.21.15.92:0/3014254083 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:08.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72d37fe700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4157b30 0x7f72d40c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.728+0000 7f72d3fff700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d41574c0 0x7f72d40ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d8c1b700 1 -- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d41574c0 msgr2=0x7f72d40ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d8c1b700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d41574c0 0x7f72d40ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d8c1b700 1 -- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4157b30 msgr2=0x7f72d40c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:08.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d8c1b700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4157b30 0x7f72d40c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d8c1b700 1 -- 172.21.15.92:0/3014254083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40b0ce0 con 0x7f72d415eda0 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d37fe700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4157b30 0x7f72d40c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d8c1b700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72d415eda0 0x7f72d415f210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f72d4071950 tx=0x7f72cc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d0ff9700 1 -- 172.21.15.92:0/3014254083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0020b0 con 0x7f72d415eda0 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d0ff9700 1 -- 172.21.15.92:0/3014254083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc003c10 con 0x7f72d415eda0 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d40b0fc0 con 0x7f72d415eda0 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72d0ff9700 1 -- 172.21.15.92:0/3014254083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00d930 con 0x7f72d415eda0 2023-06-09T04:36:08.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.729+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d4157890 con 0x7f72d415eda0 2023-06-09T04:36:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.730+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d40b11b0 con 0x7f72d415eda0 2023-06-09T04:36:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.730+0000 7f72d0ff9700 1 -- 172.21.15.92:0/3014254083 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f72cc00da90 con 0x7f72d415eda0 2023-06-09T04:36:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.731+0000 7f72d0ff9700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f72c006ace0 0x7f72c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.731+0000 7f72d0ff9700 1 -- 172.21.15.92:0/3014254083 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f72cc0892c0 con 0x7f72d415eda0 2023-06-09T04:36:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.731+0000 7f72d3fff700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f72c006ace0 0x7f72c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:08.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.731+0000 7f72d3fff700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f72c006ace0 0x7f72c006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72c80090b0 tx=0x7f72c801e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:08.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.735+0000 7f72d0ff9700 1 -- 172.21.15.92:0/3014254083 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f72cc0546c0 con 0x7f72d415eda0 2023-06-09T04:36:08.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:08.998+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f72d40740e0 con 0x7f72d415eda0 2023-06-09T04:36:09.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.002+0000 7f72d0ff9700 1 -- 172.21.15.92:0/3014254083 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f72cc057ce0 con 0x7f72d415eda0 2023-06-09T04:36:09.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f72c006ace0 msgr2=0x7f72c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f72c006ace0 0x7f72c006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72c80090b0 tx=0x7f72c801e040 comp rx=0 tx=0).stop 2023-06-09T04:36:09.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72d415eda0 msgr2=0x7f72d415f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72d415eda0 0x7f72d415f210 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f72d4071950 tx=0x7f72cc00d430 comp rx=0 tx=0).stop 2023-06-09T04:36:09.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 shutdown_connections 2023-06-09T04:36:09.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f72c006ace0 0x7f72c006d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72d415eda0 0x7f72d415f210 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f72d41574c0 0x7f72d40ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 --2- 172.21.15.92:0/3014254083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72d4157b30 0x7f72d40c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 >> 172.21.15.92:0/3014254083 conn(0x7f72d4089fe0 msgr2=0x7f72d406e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:09.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 shutdown_connections 2023-06-09T04:36:09.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.004+0000 7f72da67e700 1 -- 172.21.15.92:0/3014254083 wait complete. 2023-06-09T04:36:09.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-06-09T04:36:09.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.138+0000 7fb5e9289700 1 Processor -- start 2023-06-09T04:36:09.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.138+0000 7fb5e9289700 1 -- start start 2023-06-09T04:36:09.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e9289700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 0x7fb5e4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e9289700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e40cd0e0 con 0x7fb5e40b02d0 2023-06-09T04:36:09.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e9289700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e40aeb20 con 0x7fb5e406a950 2023-06-09T04:36:09.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e9289700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40aee00 con 0x7fb5e415c580 2023-06-09T04:36:09.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb5e40b02d0 0x7fb5e40cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37746/0 (socket says 172.21.15.92:37746) 2023-06-09T04:36:09.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e37fe700 1 -- 172.21.15.92:0/720198830 learned_addr learned my addr 172.21.15.92:0/720198830 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:09.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e2ffd700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 0x7fb5e4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795109250 0 0) 0x7fb5e40cd0e0 con 0x7fb5e40b02d0 2023-06-09T04:36:09.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5cc0035b0 con 0x7fb5e40b02d0 2023-06-09T04:36:09.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696048808 0 0) 0x7fb5e40aeb20 con 0x7fb5e406a950 2023-06-09T04:36:09.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e40cd0e0 con 0x7fb5e406a950 2023-06-09T04:36:09.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 291307303 0 0) 0x7fb5cc0035b0 con 0x7fb5e40b02d0 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 msgr2=0x7fb5e4069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 0x7fb5e4069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 >> v1:172.21.15.123:6789/0 conn(0x7fb5e406a950 legacy=0x7fb5e40cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40cf060 con 0x7fb5e40b02d0 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fb5dc0019f0 con 0x7fb5e40b02d0 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 >> v1:172.21.15.92:6789/0 conn(0x7fb5e40b02d0 legacy=0x7fb5e40cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc003bc0 0x7fb5cc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 0x7fb5cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5cc004d90 0x7fb5cc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40aee00 con 0x7fb5cc004d90 2023-06-09T04:36:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc00bba0 con 0x7fb5cc003bc0 2023-06-09T04:36:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.139+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc00bd10 con 0x7fb5e415c580 2023-06-09T04:36:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e37fe700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5cc004d90 0x7fb5cc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e27fc700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 0x7fb5cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e2ffd700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc003bc0 0x7fb5cc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e37fe700 1 -- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 msgr2=0x7fb5cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:09.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e37fe700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e415c580 0x7fb5cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e37fe700 1 -- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc003bc0 msgr2=0x7fb5cc003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:09.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e37fe700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5cc003bc0 0x7fb5cc003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e37fe700 1 -- 172.21.15.92:0/720198830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc0035b0 con 0x7fb5cc004d90 2023-06-09T04:36:09.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e37fe700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5cc004d90 0x7fb5cc005200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a6e0 tx=0x7fb5cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1209555d7d61d9ae server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0072d0 con 0x7fb5cc004d90 2023-06-09T04:36:09.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc007430 con 0x7fb5cc004d90 2023-06-09T04:36:09.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.140+0000 7fb5e1ffb700 1 -- 172.21.15.92:0/720198830 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0077c0 con 0x7fb5cc004d90 2023-06-09T04:36:09.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.141+0000 7fb5e9289700 1 -- 172.21.15.92:0/720198830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5cc004d90 msgr2=0x7fb5cc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.141+0000 7fb5e9289700 1 --2- 172.21.15.92:0/720198830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5cc004d90 0x7fb5cc005200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a6e0 tx=0x7fb5cc005880 comp rx=0 tx=0).stop 2023-06-09T04:36:09.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.141+0000 7fb5e37fe700 1 -- 172.21.15.92:0/720198830 reap_dead start 2023-06-09T04:36:09.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.141+0000 7fb5e9289700 1 -- 172.21.15.92:0/720198830 shutdown_connections 2023-06-09T04:36:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.141+0000 7fb5e9289700 1 -- 172.21.15.92:0/720198830 >> 172.21.15.92:0/720198830 conn(0x7fb5e4098a50 msgr2=0x7fb5e40cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.141+0000 7fb5e9289700 1 -- 172.21.15.92:0/720198830 shutdown_connections 2023-06-09T04:36:09.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.141+0000 7fb5e9289700 1 -- 172.21.15.92:0/720198830 wait complete. 2023-06-09T04:36:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.142+0000 7fb5e9289700 1 Processor -- start 2023-06-09T04:36:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.142+0000 7fb5e9289700 1 -- start start 2023-06-09T04:36:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e9289700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e41559f0 0x7fb5e4156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e9289700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e4155dc0 0x7fb5e40c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e9289700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40c59a0 0x7fb5e40b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e9289700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e409b370 con 0x7fb5e40c59a0 2023-06-09T04:36:09.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e9289700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40c5e20 con 0x7fb5e4155dc0 2023-06-09T04:36:09.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e9289700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e409c8d0 con 0x7fb5e41559f0 2023-06-09T04:36:09.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40c59a0 0x7fb5e40b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40c59a0 0x7fb5e40b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39426/0 (socket says 172.21.15.92:39426) 2023-06-09T04:36:09.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 -- 172.21.15.92:0/3037267923 learned_addr learned my addr 172.21.15.92:0/3037267923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e27fc700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e4155dc0 0x7fb5e40c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e2ffd700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e41559f0 0x7fb5e4156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 -- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e41559f0 msgr2=0x7fb5e4156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e41559f0 0x7fb5e4156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 -- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e4155dc0 msgr2=0x7fb5e40c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e4155dc0 0x7fb5e40c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 -- 172.21.15.92:0/3037267923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e409cbb0 con 0x7fb5e40c59a0 2023-06-09T04:36:09.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.143+0000 7fb5e37fe700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40c59a0 0x7fb5e40b40e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0057e0 tx=0x7fb5dc001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.144+0000 7fb5cb7fe700 1 -- 172.21.15.92:0/3037267923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0077a0 con 0x7fb5e40c59a0 2023-06-09T04:36:09.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.144+0000 7fb5cb7fe700 1 -- 172.21.15.92:0/3037267923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc007900 con 0x7fb5e40c59a0 2023-06-09T04:36:09.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.144+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e409ce90 con 0x7fb5e40c59a0 2023-06-09T04:36:09.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.144+0000 7fb5cb7fe700 1 -- 172.21.15.92:0/3037267923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc012400 con 0x7fb5e40c59a0 2023-06-09T04:36:09.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.144+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e409d450 con 0x7fb5e40c59a0 2023-06-09T04:36:09.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.144+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e40d0850 con 0x7fb5e40c59a0 2023-06-09T04:36:09.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.145+0000 7fb5cb7fe700 1 -- 172.21.15.92:0/3037267923 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5dc01a070 con 0x7fb5e40c59a0 2023-06-09T04:36:09.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.145+0000 7fb5cb7fe700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb5d006ace0 0x7fb5d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.145+0000 7fb5cb7fe700 1 -- 172.21.15.92:0/3037267923 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb5dc08a760 con 0x7fb5e40c59a0 2023-06-09T04:36:09.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.145+0000 7fb5e2ffd700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb5d006ace0 0x7fb5d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.146+0000 7fb5e2ffd700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb5d006ace0 0x7fb5d006d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009c40 tx=0x7fb5d40091c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.150+0000 7fb5cb7fe700 1 -- 172.21.15.92:0/3037267923 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb5dc055dd0 con 0x7fb5e40c59a0 2023-06-09T04:36:09.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.415+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fb5e40995b0 con 0x7fb5e40c59a0 2023-06-09T04:36:09.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.419+0000 7fb5cb7fe700 1 -- 172.21.15.92:0/3037267923 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb5e40995b0 con 0x7fb5e40c59a0 2023-06-09T04:36:09.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb5d006ace0 msgr2=0x7fb5d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb5d006ace0 0x7fb5d006d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009c40 tx=0x7fb5d40091c0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40c59a0 msgr2=0x7fb5e40b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40c59a0 0x7fb5e40b40e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0057e0 tx=0x7fb5dc001cb0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 shutdown_connections 2023-06-09T04:36:09.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb5d006ace0 0x7fb5d006d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e40c59a0 0x7fb5e40b40e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5e41559f0 0x7fb5e4156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 --2- 172.21.15.92:0/3037267923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e4155dc0 0x7fb5e40c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 >> 172.21.15.92:0/3037267923 conn(0x7fb5e4098a50 msgr2=0x7fb5e40ac4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 shutdown_connections 2023-06-09T04:36:09.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.421+0000 7fb5e9289700 1 -- 172.21.15.92:0/3037267923 wait complete. 2023-06-09T04:36:09.435 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:36:09.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.553+0000 7f2585c14700 1 Processor -- start 2023-06-09T04:36:09.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.553+0000 7f2585c14700 1 -- start start 2023-06-09T04:36:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f2585c14700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25800cf010 0x7f25800ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f2585c14700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25800ac400 0x7f258006e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f2585c14700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25800ce730 con 0x7f258015c7d0 2023-06-09T04:36:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f2585c14700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258006aaf0 con 0x7f25800ac400 2023-06-09T04:36:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f2585c14700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258015cba0 con 0x7f25800cf010 2023-06-09T04:36:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f257ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f25800ac400 0x7f258006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f257f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f258015c7d0 0x7f25800ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37762/0 (socket says 172.21.15.92:37762) 2023-06-09T04:36:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.554+0000 7f257f7fe700 1 -- 172.21.15.92:0/3171816579 learned_addr learned my addr 172.21.15.92:0/3171816579 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257effd700 1 --2- 172.21.15.92:0/3171816579 >> v2:172.21.15.123:3300/0 conn(0x7f25800cf010 0x7f25800ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257ffff700 1 -- 172.21.15.92:0/3171816579 >> v2:172.21.15.123:3300/0 conn(0x7f25800cf010 msgr2=0x7f25800ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:09.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257ffff700 1 --2- 172.21.15.92:0/3171816579 >> v2:172.21.15.123:3300/0 conn(0x7f25800cf010 0x7f25800ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257ffff700 1 -- 172.21.15.92:0/3171816579 >> v1:172.21.15.92:6789/0 conn(0x7f258015c7d0 legacy=0x7f25800ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257ffff700 1 -- 172.21.15.92:0/3171816579 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580069ee0 con 0x7f25800ac400 2023-06-09T04:36:09.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257effd700 1 --2- 172.21.15.92:0/3171816579 >> v2:172.21.15.123:3300/0 conn(0x7f25800cf010 0x7f25800ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257e7fc700 1 -- 172.21.15.92:0/3171816579 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626352620 0 0) 0x7f2578001290 con 0x7f258015c7d0 2023-06-09T04:36:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257ffff700 1 --2- 172.21.15.92:0/3171816579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f258006e220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2580070db0 tx=0x7f257000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e02028033f223f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257e7fc700 1 -- 172.21.15.92:0/3171816579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257000e070 con 0x7f25800ac400 2023-06-09T04:36:09.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257e7fc700 1 -- 172.21.15.92:0/3171816579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257000bbd0 con 0x7f25800ac400 2023-06-09T04:36:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f257e7fc700 1 -- 172.21.15.92:0/3171816579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570012430 con 0x7f25800ac400 2023-06-09T04:36:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f2585c14700 1 -- 172.21.15.92:0/3171816579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 msgr2=0x7f258006e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f2585c14700 1 --2- 172.21.15.92:0/3171816579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f258006e220 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2580070db0 tx=0x7f257000b330 comp rx=0 tx=0).stop 2023-06-09T04:36:09.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f2585c14700 1 -- 172.21.15.92:0/3171816579 shutdown_connections 2023-06-09T04:36:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f2585c14700 1 --2- 172.21.15.92:0/3171816579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f258006e220 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f2585c14700 1 --2- 172.21.15.92:0/3171816579 >> v2:172.21.15.123:3300/0 conn(0x7f25800cf010 0x7f25800ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f2585c14700 1 -- 172.21.15.92:0/3171816579 >> 172.21.15.92:0/3171816579 conn(0x7f2580098c10 msgr2=0x7f25800cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:09.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.555+0000 7f2585c14700 1 -- 172.21.15.92:0/3171816579 shutdown_connections 2023-06-09T04:36:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.556+0000 7f2585c14700 1 -- 172.21.15.92:0/3171816579 wait complete. 2023-06-09T04:36:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.557+0000 7f2585c14700 1 Processor -- start 2023-06-09T04:36:09.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.557+0000 7f2585c14700 1 -- start start 2023-06-09T04:36:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f25800cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25800cf010 0x7f25800c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258015c7d0 0x7f25800c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800cc8a0 con 0x7f25800ac400 2023-06-09T04:36:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801e4570 con 0x7f258015c7d0 2023-06-09T04:36:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25800c60f0 con 0x7f25800cf010 2023-06-09T04:36:09.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f25800cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f25800cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39436/0 (socket says 172.21.15.92:39436) 2023-06-09T04:36:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 -- 172.21.15.92:0/1733415564 learned_addr learned my addr 172.21.15.92:0/1733415564 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257ffff700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258015c7d0 0x7f25800c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 -- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25800cf010 msgr2=0x7f25800c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257effd700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25800cf010 0x7f25800c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25800cf010 0x7f25800c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 -- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258015c7d0 msgr2=0x7f25800c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258015c7d0 0x7f25800c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 -- 172.21.15.92:0/1733415564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2570009a70 con 0x7f25800ac400 2023-06-09T04:36:09.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257ffff700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258015c7d0 0x7f25800c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:09.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f257f7fe700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f25800cc360 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2578001eb0 tx=0x7f257800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2567fff700 1 -- 172.21.15.92:0/1733415564 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578005c40 con 0x7f25800ac400 2023-06-09T04:36:09.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2567fff700 1 -- 172.21.15.92:0/1733415564 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578005da0 con 0x7f25800ac400 2023-06-09T04:36:09.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25801e4830 con 0x7f25800ac400 2023-06-09T04:36:09.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2567fff700 1 -- 172.21.15.92:0/1733415564 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578007310 con 0x7f25800ac400 2023-06-09T04:36:09.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.558+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25801e4df0 con 0x7f25800ac400 2023-06-09T04:36:09.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.559+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25801e5010 con 0x7f25800ac400 2023-06-09T04:36:09.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.560+0000 7f2567fff700 1 -- 172.21.15.92:0/1733415564 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2578007470 con 0x7f25800ac400 2023-06-09T04:36:09.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.560+0000 7f2567fff700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f256c06ace0 0x7f256c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.560+0000 7f2567fff700 1 -- 172.21.15.92:0/1733415564 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2578088530 con 0x7f25800ac400 2023-06-09T04:36:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.560+0000 7f257effd700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f256c06ace0 0x7f256c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.560+0000 7f257effd700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f256c06ace0 0x7f256c06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2574006fd0 tx=0x7f2574008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.564+0000 7f2567fff700 1 -- 172.21.15.92:0/1733415564 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2578053930 con 0x7f25800ac400 2023-06-09T04:36:09.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.752+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25800ce730 con 0x7f25800ac400 2023-06-09T04:36:09.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.753+0000 7f2567fff700 1 -- 172.21.15.92:0/1733415564 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+14615 (secure 0 0 0) 0x7f25800ce730 con 0x7f25800ac400 2023-06-09T04:36:09.755 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:09.755 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":34,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:36:06.475866+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:36:09.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f256c06ace0 msgr2=0x7f256c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f256c06ace0 0x7f256c06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2574006fd0 tx=0x7f2574008040 comp rx=0 tx=0).stop 2023-06-09T04:36:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 msgr2=0x7f25800cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f25800cc360 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2578001eb0 tx=0x7f257800cae0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 shutdown_connections 2023-06-09T04:36:09.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f256c06ace0 0x7f256c06d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f258015c7d0 0x7f25800c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f25800ac400 0x7f25800cc360 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 --2- 172.21.15.92:0/1733415564 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f25800cf010 0x7f25800c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.755+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 >> 172.21.15.92:0/1733415564 conn(0x7f2580098c10 msgr2=0x7f258015bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:09.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.756+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 shutdown_connections 2023-06-09T04:36:09.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.756+0000 7f2585c14700 1 -- 172.21.15.92:0/1733415564 wait complete. 2023-06-09T04:36:09.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:36:09.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:36:09.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.893+0000 7f0a0e547700 1 Processor -- start 2023-06-09T04:36:09.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.893+0000 7f0a0e547700 1 -- start start 2023-06-09T04:36:09.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a0e547700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cf440 0x7f0a0815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a0e547700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0a080cce60 0x7f0a0806ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a0e547700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0815d040 con 0x7f0a08069db0 2023-06-09T04:36:09.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a0e547700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a08160ab0 con 0x7f0a080cce60 2023-06-09T04:36:09.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a0e547700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815ca10 con 0x7f0a080cf440 2023-06-09T04:36:09.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a077fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cf440 0x7f0a0815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a0cae4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0a080cce60 0x7f0a0806ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a077fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cf440 0x7f0a0815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:48356/0 (socket says 172.21.15.92:48356) 2023-06-09T04:36:09.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a0cae4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0a080cce60 0x7f0a0806ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.92:59856/0 (socket says 172.21.15.92:59856) 2023-06-09T04:36:09.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a077fe700 1 -- 172.21.15.92:0/187853995 learned_addr learned my addr 172.21.15.92:0/187853995 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:09.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.894+0000 7f0a07fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0a08069db0 0x7f0a0806a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:60272/0 (socket says 172.21.15.92:60272) 2023-06-09T04:36:09.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a0cae4700 1 -- 172.21.15.92:0/187853995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cf440 msgr2=0x7f0a0815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:09.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a0cae4700 1 --2- 172.21.15.92:0/187853995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cf440 0x7f0a0815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a0cae4700 1 -- 172.21.15.92:0/187853995 >> v1:172.21.15.123:6789/0 conn(0x7f0a08069db0 legacy=0x7f0a0806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a0cae4700 1 -- 172.21.15.92:0/187853995 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08160d50 con 0x7f0a080cce60 2023-06-09T04:36:09.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a0cae4700 1 --2- 172.21.15.92:0/187853995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cce60 0x7f0a0806ee20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009d90 tx=0x7f0a0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1964246e15972d7e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a06ffd700 1 -- 172.21.15.92:0/187853995 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0000e070 con 0x7f0a080cce60 2023-06-09T04:36:09.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a06ffd700 1 -- 172.21.15.92:0/187853995 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0000bb20 con 0x7f0a080cce60 2023-06-09T04:36:09.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.895+0000 7f0a06ffd700 1 -- 172.21.15.92:0/187853995 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00012430 con 0x7f0a080cce60 2023-06-09T04:36:09.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 -- 172.21.15.92:0/187853995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cce60 msgr2=0x7f0a0806ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 --2- 172.21.15.92:0/187853995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cce60 0x7f0a0806ee20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009d90 tx=0x7f0a0000b330 comp rx=0 tx=0).stop 2023-06-09T04:36:09.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 -- 172.21.15.92:0/187853995 shutdown_connections 2023-06-09T04:36:09.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 --2- 172.21.15.92:0/187853995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cce60 0x7f0a0806ee20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 --2- 172.21.15.92:0/187853995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cf440 0x7f0a0815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 -- 172.21.15.92:0/187853995 >> 172.21.15.92:0/187853995 conn(0x7f0a080cc380 msgr2=0x7f0a080cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:09.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 -- 172.21.15.92:0/187853995 shutdown_connections 2023-06-09T04:36:09.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.896+0000 7f0a0e547700 1 -- 172.21.15.92:0/187853995 wait complete. 2023-06-09T04:36:09.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.897+0000 7f0a0e547700 1 Processor -- start 2023-06-09T04:36:09.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.897+0000 7f0a0e547700 1 -- start start 2023-06-09T04:36:09.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a0e547700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a08069db0 0x7f0a080c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a0e547700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cce60 0x7f0a080c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a0e547700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cf440 0x7f0a080c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a0e547700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080c7920 con 0x7f0a08069db0 2023-06-09T04:36:09.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a0e547700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080c8310 con 0x7f0a080cf440 2023-06-09T04:36:09.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a0e547700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080c85f0 con 0x7f0a080cce60 2023-06-09T04:36:09.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a08069db0 0x7f0a080c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a08069db0 0x7f0a080c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39450/0 (socket says 172.21.15.92:39450) 2023-06-09T04:36:09.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 -- 172.21.15.92:0/1624326634 learned_addr learned my addr 172.21.15.92:0/1624326634 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:09.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a077fe700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cce60 0x7f0a080c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a0cae4700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cf440 0x7f0a080c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 -- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cce60 msgr2=0x7f0a080c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cce60 0x7f0a080c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 -- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cf440 msgr2=0x7f0a080c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:09.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cf440 0x7f0a080c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:09.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 -- 172.21.15.92:0/1624326634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00009a70 con 0x7f0a08069db0 2023-06-09T04:36:09.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.898+0000 7f0a07fff700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a08069db0 0x7f0a080c7cf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c520 tx=0x7f09fc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.899+0000 7f0a04ff9700 1 -- 172.21.15.92:0/1624326634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc005cb0 con 0x7f0a08069db0 2023-06-09T04:36:09.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.899+0000 7f0a04ff9700 1 -- 172.21.15.92:0/1624326634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc005e10 con 0x7f0a08069db0 2023-06-09T04:36:09.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.899+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a080c88d0 con 0x7f0a08069db0 2023-06-09T04:36:09.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.899+0000 7f0a04ff9700 1 -- 172.21.15.92:0/1624326634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0072d0 con 0x7f0a08069db0 2023-06-09T04:36:09.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.899+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a080c8e90 con 0x7f0a08069db0 2023-06-09T04:36:09.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.899+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0806be50 con 0x7f0a08069db0 2023-06-09T04:36:09.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.900+0000 7f0a04ff9700 1 -- 172.21.15.92:0/1624326634 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f09fc007430 con 0x7f0a08069db0 2023-06-09T04:36:09.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.900+0000 7f0a04ff9700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f09f406ace0 0x7f09f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:09.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.900+0000 7f0a04ff9700 1 -- 172.21.15.92:0/1624326634 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f09fc0883c0 con 0x7f0a08069db0 2023-06-09T04:36:09.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.900+0000 7f0a077fe700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f09f406ace0 0x7f09f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:09.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.901+0000 7f0a077fe700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f09f406ace0 0x7f09f406d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a0806a4d0 tx=0x7f09f80092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:09.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:09.905+0000 7f0a04ff9700 1 -- 172.21.15.92:0/1624326634 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f09fc053870 con 0x7f0a08069db0 2023-06-09T04:36:10.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.092+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a080c9080 con 0x7f0a08069db0 2023-06-09T04:36:10.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.092+0000 7f0a04ff9700 1 -- 172.21.15.92:0/1624326634 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09fc056e90 con 0x7f0a08069db0 2023-06-09T04:36:10.094 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:36:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f09f406ace0 msgr2=0x7f09f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f09f406ace0 0x7f09f406d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a0806a4d0 tx=0x7f09f80092c0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a08069db0 msgr2=0x7f0a080c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a08069db0 0x7f0a080c7cf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c520 tx=0x7f09fc00c830 comp rx=0 tx=0).stop 2023-06-09T04:36:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 shutdown_connections 2023-06-09T04:36:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f09f406ace0 0x7f09f406d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a080cf440 0x7f0a080c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a08069db0 0x7f0a080c7cf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 --2- 172.21.15.92:0/1624326634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0a080cce60 0x7f0a080c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 >> 172.21.15.92:0/1624326634 conn(0x7f0a080cc380 msgr2=0x7f0a080cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 shutdown_connections 2023-06-09T04:36:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.094+0000 7f0a0e547700 1 -- 172.21.15.92:0/1624326634 wait complete. 2023-06-09T04:36:10.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:36:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.229+0000 7f0de493e700 1 Processor -- start 2023-06-09T04:36:10.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.229+0000 7f0de493e700 1 -- start start 2023-06-09T04:36:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0de493e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0de00ce3d0 0x7f0de00713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0de493e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 0x7f0de015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0de493e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de00ce190 con 0x7f0de00aed50 2023-06-09T04:36:10.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0de493e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de0072350 con 0x7f0de00ce3d0 2023-06-09T04:36:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0de493e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de0071ab0 con 0x7f0de015d470 2023-06-09T04:36:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0dddd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0de00aed50 0x7f0de00ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37790/0 (socket says 172.21.15.92:37790) 2023-06-09T04:36:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0dddd9b700 1 -- 172.21.15.92:0/1319870857 learned_addr learned my addr 172.21.15.92:0/1319870857 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556725952 0 0) 0x7f0de00ce190 con 0x7f0de00aed50 2023-06-09T04:36:10.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0dded9d700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 0x7f0de015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc80035b0 con 0x7f0de00aed50 2023-06-09T04:36:10.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0dde59c700 1 --2- 172.21.15.92:0/1319870857 >> v2:172.21.15.123:3300/0 conn(0x7f0de00ce3d0 0x7f0de00713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3641259739 0 0) 0x7f0dc80035b0 con 0x7f0de00aed50 2023-06-09T04:36:10.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 msgr2=0x7f0de015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 0x7f0de015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 >> v2:172.21.15.123:3300/0 conn(0x7f0de00ce3d0 msgr2=0x7f0de00713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 --2- 172.21.15.92:0/1319870857 >> v2:172.21.15.123:3300/0 conn(0x7f0de00ce3d0 0x7f0de00713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de0071d90 con 0x7f0de00aed50 2023-06-09T04:36:10.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f0dd4001670 con 0x7f0de00aed50 2023-06-09T04:36:10.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 >> v1:172.21.15.92:6789/0 conn(0x7f0de00aed50 legacy=0x7f0de00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8003a00 0x7f0dc8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.230+0000 7f0ddd59a700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dc8004430 0x7f0dc80048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0ddd59a700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 0x7f0dc8004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de0072350 con 0x7f0dc8004430 2023-06-09T04:36:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de0071ab0 con 0x7f0dc8003a00 2023-06-09T04:36:10.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc800ba10 con 0x7f0de015d470 2023-06-09T04:36:10.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dddd9b700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dc8004430 0x7f0dc80048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dded9d700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 0x7f0dc8004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dde59c700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8003a00 0x7f0dc8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dddd9b700 1 -- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 msgr2=0x7f0dc8004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dddd9b700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0de015d470 0x7f0dc8004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dddd9b700 1 -- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8003a00 msgr2=0x7f0dc8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dddd9b700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0dc8003a00 0x7f0dc8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dddd9b700 1 -- 172.21.15.92:0/1319870857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80035b0 con 0x7f0dc8004430 2023-06-09T04:36:10.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0dddd9b700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dc8004430 0x7f0dc80048f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400c320 tx=0x7f0dc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4583ae727a8f822 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400ce60 con 0x7f0dc8004430 2023-06-09T04:36:10.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd4005960 con 0x7f0dc8004430 2023-06-09T04:36:10.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.231+0000 7f0ddd59a700 1 -- 172.21.15.92:0/1319870857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4005cf0 con 0x7f0dc8004430 2023-06-09T04:36:10.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.232+0000 7f0de493e700 1 -- 172.21.15.92:0/1319870857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dc8004430 msgr2=0x7f0dc80048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.232+0000 7f0de493e700 1 --2- 172.21.15.92:0/1319870857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0dc8004430 0x7f0dc80048f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400c320 tx=0x7f0dc8005880 comp rx=0 tx=0).stop 2023-06-09T04:36:10.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.232+0000 7f0dded9d700 1 -- 172.21.15.92:0/1319870857 reap_dead start 2023-06-09T04:36:10.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.232+0000 7f0de493e700 1 -- 172.21.15.92:0/1319870857 shutdown_connections 2023-06-09T04:36:10.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.232+0000 7f0de493e700 1 -- 172.21.15.92:0/1319870857 >> 172.21.15.92:0/1319870857 conn(0x7f0de00cc380 msgr2=0x7f0de00cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:10.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.232+0000 7f0de493e700 1 -- 172.21.15.92:0/1319870857 shutdown_connections 2023-06-09T04:36:10.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.232+0000 7f0de493e700 1 -- 172.21.15.92:0/1319870857 wait complete. 2023-06-09T04:36:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.233+0000 7f0de493e700 1 Processor -- start 2023-06-09T04:36:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.233+0000 7f0de493e700 1 -- start start 2023-06-09T04:36:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0de493e700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8004430 0x7f0de014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0de493e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de0069db0 0x7f0de006a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0de493e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0de00ad570 0x7f0de00ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0de493e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de014d240 con 0x7f0de0069db0 2023-06-09T04:36:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0de493e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de006a760 con 0x7f0de00ad570 2023-06-09T04:36:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0de493e700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de00ac3a0 con 0x7f0dc8004430 2023-06-09T04:36:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de0069db0 0x7f0de006a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de0069db0 0x7f0de006a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39472/0 (socket says 172.21.15.92:39472) 2023-06-09T04:36:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 -- 172.21.15.92:0/60384727 learned_addr learned my addr 172.21.15.92:0/60384727 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dded9d700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0de00ad570 0x7f0de00ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 -- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8004430 msgr2=0x7f0de014cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dde59c700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8004430 0x7f0de014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8004430 0x7f0de014cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 -- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0de00ad570 msgr2=0x7f0de00ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0de00ad570 0x7f0de00ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dddd9b700 1 -- 172.21.15.92:0/60384727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de00aba70 con 0x7f0de0069db0 2023-06-09T04:36:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.234+0000 7f0dded9d700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0de00ad570 0x7f0de00ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:10.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.235+0000 7f0dddd9b700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de0069db0 0x7f0de006a220 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80056d0 tx=0x7f0dd4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.235+0000 7f0dceffd700 1 -- 172.21.15.92:0/60384727 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd40016a0 con 0x7f0de0069db0 2023-06-09T04:36:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.235+0000 7f0dceffd700 1 -- 172.21.15.92:0/60384727 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd4004a80 con 0x7f0de0069db0 2023-06-09T04:36:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.235+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de00abd50 con 0x7f0de0069db0 2023-06-09T04:36:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.235+0000 7f0dceffd700 1 -- 172.21.15.92:0/60384727 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4004e10 con 0x7f0de0069db0 2023-06-09T04:36:10.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.235+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de00ac830 con 0x7f0de0069db0 2023-06-09T04:36:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.236+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de004e8e0 con 0x7f0de0069db0 2023-06-09T04:36:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.236+0000 7f0dceffd700 1 -- 172.21.15.92:0/60384727 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0dd40070d0 con 0x7f0de0069db0 2023-06-09T04:36:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.236+0000 7f0dceffd700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dc006ace0 0x7f0dc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.237+0000 7f0dceffd700 1 -- 172.21.15.92:0/60384727 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0dd408a8a0 con 0x7f0de0069db0 2023-06-09T04:36:10.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.237+0000 7f0dde59c700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dc006ace0 0x7f0dc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.237+0000 7f0dde59c700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dc006ace0 0x7f0dc006d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009a20 tx=0x7f0dd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.241+0000 7f0dceffd700 1 -- 172.21.15.92:0/60384727 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0dd4055e50 con 0x7f0de0069db0 2023-06-09T04:36:10.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.427+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0de00b2120 con 0x7f0de0069db0 2023-06-09T04:36:10.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.428+0000 7f0dceffd700 1 -- 172.21.15.92:0/60384727 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0dd4007da0 con 0x7f0de0069db0 2023-06-09T04:36:10.430 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:36:10.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dc006ace0 msgr2=0x7f0dc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dc006ace0 0x7f0dc006d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009a20 tx=0x7f0dd0008040 comp rx=0 tx=0).stop 2023-06-09T04:36:10.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de0069db0 msgr2=0x7f0de006a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de0069db0 0x7f0de006a220 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80056d0 tx=0x7f0dd4001a00 comp rx=0 tx=0).stop 2023-06-09T04:36:10.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 shutdown_connections 2023-06-09T04:36:10.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f0dc006ace0 0x7f0dc006d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0de00ad570 0x7f0de00ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0dc8004430 0x7f0de014cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 --2- 172.21.15.92:0/60384727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de0069db0 0x7f0de006a220 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 >> 172.21.15.92:0/60384727 conn(0x7f0de00cc380 msgr2=0x7f0de00d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:10.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 shutdown_connections 2023-06-09T04:36:10.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.430+0000 7f0de493e700 1 -- 172.21.15.92:0/60384727 wait complete. 2023-06-09T04:36:10.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-09T04:36:10.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.562+0000 7f9df0c02700 1 Processor -- start 2023-06-09T04:36:10.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.562+0000 7f9df0c02700 1 -- start start 2023-06-09T04:36:10.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9df0c02700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 0x7f9dec0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9df0c02700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec0cdd10 con 0x7f9dec0afef0 2023-06-09T04:36:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9df0c02700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec15a750 con 0x7f9dec0af9a0 2023-06-09T04:36:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9df0c02700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec06eca0 con 0x7f9dec15a380 2023-06-09T04:36:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9dec0afef0 0x7f9dec06f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37810/0 (socket says 172.21.15.92:37810) 2023-06-09T04:36:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de9d9b700 1 -- 172.21.15.92:0/2193355681 learned_addr learned my addr 172.21.15.92:0/2193355681 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298629261 0 0) 0x7f9dec0cdd10 con 0x7f9dec0afef0 2023-06-09T04:36:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9dea59c700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 0x7f9dec0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd40035b0 con 0x7f9dec0afef0 2023-06-09T04:36:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009752694 0 0) 0x7f9dec15a750 con 0x7f9dec0af9a0 2023-06-09T04:36:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dec0cdd10 con 0x7f9dec0af9a0 2023-06-09T04:36:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780772886 0 0) 0x7f9dd40035b0 con 0x7f9dec0afef0 2023-06-09T04:36:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 msgr2=0x7f9dec0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 0x7f9dec0ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 >> v1:172.21.15.123:6789/0 conn(0x7f9dec0af9a0 legacy=0x7f9dec06f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0ce270 con 0x7f9dec0afef0 2023-06-09T04:36:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f9de0001670 con 0x7f9dec0afef0 2023-06-09T04:36:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 >> v1:172.21.15.92:6789/0 conn(0x7f9dec0afef0 legacy=0x7f9dec06f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.563+0000 7f9de959a700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd4003bc0 0x7f9dd4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 0x7f9dd4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dd4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec06eca0 con 0x7f9dd4004d90 2023-06-09T04:36:10.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400bba0 con 0x7f9dd4003bc0 2023-06-09T04:36:10.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400bd60 con 0x7f9dec15a380 2023-06-09T04:36:10.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dead9d700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dd4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dea59c700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd4003bc0 0x7f9dd4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de9d9b700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 0x7f9dd4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dead9d700 1 -- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 msgr2=0x7f9dd4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dead9d700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec15a380 0x7f9dd4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dead9d700 1 -- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd4003bc0 msgr2=0x7f9dd4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dead9d700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dd4003bc0 0x7f9dd4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dead9d700 1 -- 172.21.15.92:0/2193355681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd40035b0 con 0x7f9dd4004d90 2023-06-09T04:36:10.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9dead9d700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dd4005200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c2f0 tx=0x7f9dd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08d182b3c2ac64f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4001f30 con 0x7f9dd4004d90 2023-06-09T04:36:10.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de400ce90 con 0x7f9dd4004d90 2023-06-09T04:36:10.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.564+0000 7f9de959a700 1 -- 172.21.15.92:0/2193355681 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4003c40 con 0x7f9dd4004d90 2023-06-09T04:36:10.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.565+0000 7f9df0c02700 1 -- 172.21.15.92:0/2193355681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 msgr2=0x7f9dd4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.565+0000 7f9df0c02700 1 --2- 172.21.15.92:0/2193355681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dd4005200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c2f0 tx=0x7f9dd4005880 comp rx=0 tx=0).stop 2023-06-09T04:36:10.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.565+0000 7f9dead9d700 1 -- 172.21.15.92:0/2193355681 reap_dead start 2023-06-09T04:36:10.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.565+0000 7f9df0c02700 1 -- 172.21.15.92:0/2193355681 shutdown_connections 2023-06-09T04:36:10.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.565+0000 7f9df0c02700 1 -- 172.21.15.92:0/2193355681 >> 172.21.15.92:0/2193355681 conn(0x7f9dec089fe0 msgr2=0x7f9dec0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:10.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.565+0000 7f9df0c02700 1 -- 172.21.15.92:0/2193355681 shutdown_connections 2023-06-09T04:36:10.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.565+0000 7f9df0c02700 1 -- 172.21.15.92:0/2193355681 wait complete. 2023-06-09T04:36:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.566+0000 7f9df0c02700 1 Processor -- start 2023-06-09T04:36:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.567+0000 7f9df0c02700 1 -- start start 2023-06-09T04:36:10.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.567+0000 7f9df0c02700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dec152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.567+0000 7f9df0c02700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec0b03d0 0x7f9dec0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.567+0000 7f9df0c02700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec0b2970 0x7f9dec0b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.567+0000 7f9df0c02700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0b3320 con 0x7f9dd4004d90 2023-06-09T04:36:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.567+0000 7f9df0c02700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0b1b50 con 0x7f9dec0b03d0 2023-06-09T04:36:10.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.567+0000 7f9df0c02700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0b0f40 con 0x7f9dec0b2970 2023-06-09T04:36:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dec152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dec152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39488/0 (socket says 172.21.15.92:39488) 2023-06-09T04:36:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 -- 172.21.15.92:0/3165945588 learned_addr learned my addr 172.21.15.92:0/3165945588 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dead9d700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec0b2970 0x7f9dec0b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9de9d9b700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec0b03d0 0x7f9dec0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 -- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec0b2970 msgr2=0x7f9dec0b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec0b2970 0x7f9dec0b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 -- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec0b03d0 msgr2=0x7f9dec0b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec0b03d0 0x7f9dec0b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 -- 172.21.15.92:0/3165945588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0b1220 con 0x7f9dd4004d90 2023-06-09T04:36:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9de9d9b700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec0b03d0 0x7f9dec0b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9dea59c700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dec152020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc011aa0 tx=0x7f9ddc011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9ddaffd700 1 -- 172.21.15.92:0/3165945588 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00f980 con 0x7f9dd4004d90 2023-06-09T04:36:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9ddaffd700 1 -- 172.21.15.92:0/3165945588 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc00fae0 con 0x7f9dd4004d90 2023-06-09T04:36:10.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec0b1500 con 0x7f9dd4004d90 2023-06-09T04:36:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9ddaffd700 1 -- 172.21.15.92:0/3165945588 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00fe90 con 0x7f9dd4004d90 2023-06-09T04:36:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.568+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec0c90e0 con 0x7f9dd4004d90 2023-06-09T04:36:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.569+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec0b1e30 con 0x7f9dd4004d90 2023-06-09T04:36:10.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.570+0000 7f9ddaffd700 1 -- 172.21.15.92:0/3165945588 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ddc014440 con 0x7f9dd4004d90 2023-06-09T04:36:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.570+0000 7f9ddaffd700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9dcc06ace0 0x7f9dcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.570+0000 7f9ddaffd700 1 -- 172.21.15.92:0/3165945588 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9ddc089930 con 0x7f9dd4004d90 2023-06-09T04:36:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.570+0000 7f9de9d9b700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9dcc06ace0 0x7f9dcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.570+0000 7f9de9d9b700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9dcc06ace0 0x7f9dcc06d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9dec1560c0 tx=0x7f9de0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.574+0000 7f9ddaffd700 1 -- 172.21.15.92:0/3165945588 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9ddc058930 con 0x7f9dd4004d90 2023-06-09T04:36:10.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.761+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9dec0740e0 con 0x7f9dd4004d90 2023-06-09T04:36:10.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.762+0000 7f9ddaffd700 1 -- 172.21.15.92:0/3165945588 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9ddc0584c0 con 0x7f9dd4004d90 2023-06-09T04:36:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:10.764 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-06-09T04:36:10.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-06-09T04:36:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-06-09T04:36:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-06-09T04:36:10.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-06-09T04:36:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-06-09T04:36:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-06-09T04:36:10.766 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-06-09T04:36:10.766 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-06-09T04:36:10.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9dcc06ace0 msgr2=0x7f9dcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9dcc06ace0 0x7f9dcc06d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9dec1560c0 tx=0x7f9de0009040 comp rx=0 tx=0).stop 2023-06-09T04:36:10.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 msgr2=0x7f9dec152020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dec152020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc011aa0 tx=0x7f9ddc011e60 comp rx=0 tx=0).stop 2023-06-09T04:36:10.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 shutdown_connections 2023-06-09T04:36:10.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f9dcc06ace0 0x7f9dcc06d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9dec0b2970 0x7f9dec0b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4004d90 0x7f9dec152020 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 --2- 172.21.15.92:0/3165945588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9dec0b03d0 0x7f9dec0b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 >> 172.21.15.92:0/3165945588 conn(0x7f9dec089fe0 msgr2=0x7f9dec0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:10.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 shutdown_connections 2023-06-09T04:36:10.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.764+0000 7f9df0c02700 1 -- 172.21.15.92:0/3165945588 wait complete. 2023-06-09T04:36:10.777 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-09T04:36:10.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-09T04:36:10.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.901+0000 7fa411409700 1 Processor -- start 2023-06-09T04:36:10.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.901+0000 7fa411409700 1 -- start start 2023-06-09T04:36:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa411409700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa411409700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa40c15a7e0 0x7fa40c0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa411409700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa40c0ce580 0x7fa40c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa411409700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0cef60 con 0x7fa40c0ce580 2023-06-09T04:36:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa411409700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0d1190 con 0x7fa40c15a7e0 2023-06-09T04:36:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa411409700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0d1470 con 0x7fa40c15a410 2023-06-09T04:36:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa40c0ce580 0x7fa40c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa40c0ce580 0x7fa40c0cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39498/0 (socket says 172.21.15.92:39498) 2023-06-09T04:36:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 -- 172.21.15.92:0/872768530 learned_addr learned my addr 172.21.15.92:0/872768530 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40a7fc700 1 --2- 172.21.15.92:0/872768530 >> v2:172.21.15.123:3300/0 conn(0x7fa40c15a7e0 0x7fa40c0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40affd700 1 --2- 172.21.15.92:0/872768530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 -- 172.21.15.92:0/872768530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 msgr2=0x7fa40c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 --2- 172.21.15.92:0/872768530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c06fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 -- 172.21.15.92:0/872768530 >> v2:172.21.15.123:3300/0 conn(0x7fa40c15a7e0 msgr2=0x7fa40c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 --2- 172.21.15.92:0/872768530 >> v2:172.21.15.123:3300/0 conn(0x7fa40c15a7e0 0x7fa40c0d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 -- 172.21.15.92:0/872768530 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0ad580 con 0x7fa40c0ce580 2023-06-09T04:36:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40a7fc700 1 --2- 172.21.15.92:0/872768530 >> v2:172.21.15.123:3300/0 conn(0x7fa40c15a7e0 0x7fa40c0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:10.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.902+0000 7fa40b7fe700 1 --2- 172.21.15.92:0/872768530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0cea20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0721b0 tx=0x7fa40400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e1ad5582ef876c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa409ffb700 1 -- 172.21.15.92:0/872768530 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40400e070 con 0x7fa40c0ce580 2023-06-09T04:36:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa409ffb700 1 -- 172.21.15.92:0/872768530 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40400b9c0 con 0x7fa40c0ce580 2023-06-09T04:36:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa409ffb700 1 -- 172.21.15.92:0/872768530 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40400bd70 con 0x7fa40c0ce580 2023-06-09T04:36:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 -- 172.21.15.92:0/872768530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 msgr2=0x7fa40c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 --2- 172.21.15.92:0/872768530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0cea20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0721b0 tx=0x7fa40400b330 comp rx=0 tx=0).stop 2023-06-09T04:36:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 -- 172.21.15.92:0/872768530 shutdown_connections 2023-06-09T04:36:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 --2- 172.21.15.92:0/872768530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0cea20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 --2- 172.21.15.92:0/872768530 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 --2- 172.21.15.92:0/872768530 >> v2:172.21.15.123:3300/0 conn(0x7fa40c15a7e0 0x7fa40c0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 -- 172.21.15.92:0/872768530 >> 172.21.15.92:0/872768530 conn(0x7fa40c066370 msgr2=0x7fa40c099d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 -- 172.21.15.92:0/872768530 shutdown_connections 2023-06-09T04:36:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.903+0000 7fa411409700 1 -- 172.21.15.92:0/872768530 wait complete. 2023-06-09T04:36:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 Processor -- start 2023-06-09T04:36:10.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 -- start start 2023-06-09T04:36:10.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c15a7e0 0x7fa40c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0af620 con 0x7fa40c0ce580 2023-06-09T04:36:10.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0b66a0 con 0x7fa40c15a7e0 2023-06-09T04:36:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa411409700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c15ac00 con 0x7fa40c15a410 2023-06-09T04:36:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa40affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa40affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39502/0 (socket says 172.21.15.92:39502) 2023-06-09T04:36:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.905+0000 7fa40affd700 1 -- 172.21.15.92:0/3723438181 learned_addr learned my addr 172.21.15.92:0/3723438181 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:10.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40a7fc700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40b7fe700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c15a7e0 0x7fa40c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40affd700 1 -- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 msgr2=0x7fa40c0b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40affd700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c0b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40affd700 1 -- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c15a7e0 msgr2=0x7fa40c0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:10.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40affd700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c15a7e0 0x7fa40c0b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40affd700 1 -- 172.21.15.92:0/3723438181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa404009b30 con 0x7fa40c0ce580 2023-06-09T04:36:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40b7fe700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c15a7e0 0x7fa40c0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa40affd700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0b41d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e810 tx=0x7fa3fc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa3f37fe700 1 -- 172.21.15.92:0/3723438181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc013610 con 0x7fa40c0ce580 2023-06-09T04:36:10.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa3f37fe700 1 -- 172.21.15.92:0/3723438181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc013770 con 0x7fa40c0ce580 2023-06-09T04:36:10.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c15aee0 con 0x7fa40c0ce580 2023-06-09T04:36:10.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa3f37fe700 1 -- 172.21.15.92:0/3723438181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc013b00 con 0x7fa40c0ce580 2023-06-09T04:36:10.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.906+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c15b4a0 con 0x7fa40c0ce580 2023-06-09T04:36:10.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.907+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c15b6c0 con 0x7fa40c0ce580 2023-06-09T04:36:10.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.908+0000 7fa3f37fe700 1 -- 172.21.15.92:0/3723438181 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3fc013c60 con 0x7fa40c0ce580 2023-06-09T04:36:10.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.908+0000 7fa3f37fe700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3f806ace0 0x7fa3f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:10.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.908+0000 7fa3f37fe700 1 -- 172.21.15.92:0/3723438181 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa3fc088c80 con 0x7fa40c0ce580 2023-06-09T04:36:10.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.908+0000 7fa40a7fc700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3f806ace0 0x7fa3f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:10.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.908+0000 7fa40a7fc700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3f806ace0 0x7fa3f806d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0b3920 tx=0x7fa400008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:10.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:10.912+0000 7fa3f37fe700 1 -- 172.21.15.92:0/3723438181 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3fc054110 con 0x7fa40c0ce580 2023-06-09T04:36:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.099+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa40c06f810 con 0x7fa40c0ce580 2023-06-09T04:36:11.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.423+0000 7fa3f37fe700 1 -- 172.21.15.92:0/3723438181 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa3fc0210b0 con 0x7fa40c0ce580 2023-06-09T04:36:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.426+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3f806ace0 msgr2=0x7fa3f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:11.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.426+0000 7fa411409700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3f806ace0 0x7fa3f806d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0b3920 tx=0x7fa400008040 comp rx=0 tx=0).stop 2023-06-09T04:36:11.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.426+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 msgr2=0x7fa40c0b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:11.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.426+0000 7fa411409700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0b41d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e810 tx=0x7fa3fc00eb20 comp rx=0 tx=0).stop 2023-06-09T04:36:11.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.427+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 shutdown_connections 2023-06-09T04:36:11.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.427+0000 7fa411409700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa3f806ace0 0x7fa3f806d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.427+0000 7fa411409700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa40c15a7e0 0x7fa40c0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.427+0000 7fa411409700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa40c0ce580 0x7fa40c0b41d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.427+0000 7fa411409700 1 --2- 172.21.15.92:0/3723438181 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa40c15a410 0x7fa40c0b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.427+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 >> 172.21.15.92:0/3723438181 conn(0x7fa40c066370 msgr2=0x7fa40c159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:11.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.427+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 shutdown_connections 2023-06-09T04:36:11.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.428+0000 7fa411409700 1 -- 172.21.15.92:0/3723438181 wait complete. 2023-06-09T04:36:11.432 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-06-09T04:36:11.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-09T04:36:11.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.572+0000 7fb34b06f700 1 Processor -- start 2023-06-09T04:36:11.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.572+0000 7fb34b06f700 1 -- start start 2023-06-09T04:36:11.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb34b06f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 0x7fb34415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb34b06f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34415d040 con 0x7fb3440cf440 2023-06-09T04:36:11.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb34b06f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb344160ab0 con 0x7fb3440cce60 2023-06-09T04:36:11.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb34b06f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34415ca10 con 0x7fb344069db0 2023-06-09T04:36:11.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb343fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3440cf440 0x7fb34406a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37842/0 (socket says 172.21.15.92:37842) 2023-06-09T04:36:11.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb343fff700 1 -- 172.21.15.92:0/1846571933 learned_addr learned my addr 172.21.15.92:0/1846571933 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:11.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb348e0b700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 0x7fb34415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379050822 0 0) 0x7fb34415d040 con 0x7fb3440cf440 2023-06-09T04:36:11.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3340035b0 con 0x7fb3440cf440 2023-06-09T04:36:11.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431086952 0 0) 0x7fb3340035b0 con 0x7fb3440cf440 2023-06-09T04:36:11.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 msgr2=0x7fb34415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:11.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 0x7fb34415d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 >> v1:172.21.15.123:6789/0 conn(0x7fb3440cce60 legacy=0x7fb34406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb344160d50 con 0x7fb3440cf440 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fb330001670 con 0x7fb3440cf440 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 >> v1:172.21.15.92:6789/0 conn(0x7fb3440cf440 legacy=0x7fb34406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb334003a30 0x7fb334003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334004850 0x7fb334004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 0x7fb334005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34415ca10 con 0x7fb334003a30 2023-06-09T04:36:11.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33400bb10 con 0x7fb334004850 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33400bcd0 con 0x7fb344069db0 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb348e0b700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb334003a30 0x7fb334003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.573+0000 7fb343fff700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334004850 0x7fb334004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34960c700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 0x7fb334005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb348e0b700 1 -- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 msgr2=0x7fb334005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb348e0b700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb344069db0 0x7fb334005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb348e0b700 1 -- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334004850 msgr2=0x7fb334004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:11.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb348e0b700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334004850 0x7fb334004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb348e0b700 1 -- 172.21.15.92:0/1846571933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340035b0 con 0x7fb334003a30 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb348e0b700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb334003a30 0x7fb334003e40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb338009b60 tx=0x7fb334005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db783e57f8bf7a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800e6c0 con 0x7fb334003a30 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800e820 con 0x7fb334003a30 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb3437fe700 1 -- 172.21.15.92:0/1846571933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800eb90 con 0x7fb334003a30 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34b06f700 1 -- 172.21.15.92:0/1846571933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb334003a30 msgr2=0x7fb334003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34b06f700 1 --2- 172.21.15.92:0/1846571933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb334003a30 0x7fb334003e40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb338009b60 tx=0x7fb334005880 comp rx=0 tx=0).stop 2023-06-09T04:36:11.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34960c700 1 -- 172.21.15.92:0/1846571933 reap_dead start 2023-06-09T04:36:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34b06f700 1 -- 172.21.15.92:0/1846571933 shutdown_connections 2023-06-09T04:36:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34b06f700 1 -- 172.21.15.92:0/1846571933 >> 172.21.15.92:0/1846571933 conn(0x7fb3440cc380 msgr2=0x7fb3440cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34b06f700 1 -- 172.21.15.92:0/1846571933 shutdown_connections 2023-06-09T04:36:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.574+0000 7fb34b06f700 1 -- 172.21.15.92:0/1846571933 wait complete. 2023-06-09T04:36:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.576+0000 7fb34b06f700 1 Processor -- start 2023-06-09T04:36:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.576+0000 7fb34b06f700 1 -- start start 2023-06-09T04:36:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334003a30 0x7fb3440c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3440b1780 0x7fb344155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3440b1bf0 0x7fb3441561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb344156710 con 0x7fb3440b1bf0 2023-06-09T04:36:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c8d50 con 0x7fb334003a30 2023-06-09T04:36:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440c8440 con 0x7fb3440b1780 2023-06-09T04:36:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3440b1bf0 0x7fb3441561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3440b1bf0 0x7fb3441561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39524/0 (socket says 172.21.15.92:39524) 2023-06-09T04:36:11.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 -- 172.21.15.92:0/3988174308 learned_addr learned my addr 172.21.15.92:0/3988174308 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb348e0b700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334003a30 0x7fb3440c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb343fff700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3440b1780 0x7fb344155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 -- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3440b1780 msgr2=0x7fb344155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3440b1780 0x7fb344155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 -- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334003a30 msgr2=0x7fb3440c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334003a30 0x7fb3440c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 -- 172.21.15.92:0/3988174308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440b35b0 con 0x7fb3440b1bf0 2023-06-09T04:36:11.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34960c700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3440b1bf0 0x7fb3441561d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb33c000f80 tx=0x7fb33c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb3417fa700 1 -- 172.21.15.92:0/3988174308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c003c10 con 0x7fb3440b1bf0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb3417fa700 1 -- 172.21.15.92:0/3988174308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c003d70 con 0x7fb3440b1bf0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3440c8700 con 0x7fb3440b1bf0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb3417fa700 1 -- 172.21.15.92:0/3988174308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c005310 con 0x7fb3440b1bf0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.577+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3440c9720 con 0x7fb3440b1bf0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.578+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3440c98e0 con 0x7fb3440b1bf0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.579+0000 7fb3417fa700 1 -- 172.21.15.92:0/3988174308 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb33c005470 con 0x7fb3440b1bf0 2023-06-09T04:36:11.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.579+0000 7fb3417fa700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb32806ace0 0x7fb32806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.579+0000 7fb3417fa700 1 -- 172.21.15.92:0/3988174308 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fb33c0891e0 con 0x7fb3440b1bf0 2023-06-09T04:36:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.579+0000 7fb348e0b700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb32806ace0 0x7fb32806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.580+0000 7fb348e0b700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb32806ace0 0x7fb32806d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3380090b0 tx=0x7fb33801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:11.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.584+0000 7fb3417fa700 1 -- 172.21.15.92:0/3988174308 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb33c054460 con 0x7fb3440b1bf0 2023-06-09T04:36:11.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:11.773+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb3440c8ee0 con 0x7fb3440b1bf0 2023-06-09T04:36:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.428+0000 7fb3417fa700 1 -- 172.21.15.92:0/3988174308 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb33c057a80 con 0x7fb3440b1bf0 2023-06-09T04:36:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.434+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb32806ace0 msgr2=0x7fb32806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.434+0000 7fb34b06f700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb32806ace0 0x7fb32806d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb3380090b0 tx=0x7fb33801f040 comp rx=0 tx=0).stop 2023-06-09T04:36:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.435+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3440b1bf0 msgr2=0x7fb3441561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.435+0000 7fb34b06f700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3440b1bf0 0x7fb3441561d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb33c000f80 tx=0x7fb33c00edd0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 shutdown_connections 2023-06-09T04:36:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fb32806ace0 0x7fb32806d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3440b1bf0 0x7fb3441561d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb334003a30 0x7fb3440c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 --2- 172.21.15.92:0/3988174308 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3440b1780 0x7fb344155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 >> 172.21.15.92:0/3988174308 conn(0x7fb3440cc380 msgr2=0x7fb34415fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:12.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 shutdown_connections 2023-06-09T04:36:12.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.437+0000 7fb34b06f700 1 -- 172.21.15.92:0/3988174308 wait complete. 2023-06-09T04:36:12.441 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-06-09T04:36:12.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-09T04:36:12.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.582+0000 7fdfe96f0700 1 Processor -- start 2023-06-09T04:36:12.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.582+0000 7fdfe96f0700 1 -- start start 2023-06-09T04:36:12.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe96f0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdfe415ced0 0x7fdfe40cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe96f0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe40d2b00 con 0x7fdfe415d2a0 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe96f0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe40d39d0 con 0x7fdfe40cf670 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe96f0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe40cd3c0 con 0x7fdfe415ced0 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe27fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdfe415d2a0 0x7fdfe40d32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37860/0 (socket says 172.21.15.92:37860) 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe27fc700 1 -- 172.21.15.92:0/3043628860 learned_addr learned my addr 172.21.15.92:0/3043628860 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe2ffd700 1 --2- 172.21.15.92:0/3043628860 >> v2:172.21.15.123:3300/0 conn(0x7fdfe415ced0 0x7fdfe40cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020383010 0 0) 0x7fdfe40d2b00 con 0x7fdfe415d2a0 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfcc0035b0 con 0x7fdfe415d2a0 2023-06-09T04:36:12.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2196232374 0 0) 0x7fdfcc0035b0 con 0x7fdfe415d2a0 2023-06-09T04:36:12.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 >> v2:172.21.15.123:3300/0 conn(0x7fdfe415ced0 msgr2=0x7fdfe40cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:12.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 --2- 172.21.15.92:0/3043628860 >> v2:172.21.15.123:3300/0 conn(0x7fdfe415ced0 0x7fdfe40cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 >> v1:172.21.15.123:6789/0 conn(0x7fdfe40cf670 legacy=0x7fdfe40cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe40cd6a0 con 0x7fdfe415d2a0 2023-06-09T04:36:12.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fdfd4001670 con 0x7fdfe415d2a0 2023-06-09T04:36:12.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 >> v1:172.21.15.92:6789/0 conn(0x7fdfe415d2a0 legacy=0x7fdfe40d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc003a00 0x7fdfcc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.583+0000 7fdfe1ffb700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfcc004850 0x7fdfcc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe1ffb700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc005200 0x7fdfcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40cd3c0 con 0x7fdfcc005200 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc00c080 con 0x7fdfcc003a00 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc00c240 con 0x7fdfcc004850 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc005200 0x7fdfcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe2ffd700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc003a00 0x7fdfcc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe27fc700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfcc004850 0x7fdfcc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 -- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfcc004850 msgr2=0x7fdfcc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:12.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfcc004850 0x7fdfcc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 -- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc003a00 msgr2=0x7fdfcc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:12.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc003a00 0x7fdfcc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 -- 172.21.15.92:0/3043628860 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc0035b0 con 0x7fdfcc005200 2023-06-09T04:36:12.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe2ffd700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc003a00 0x7fdfcc003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:36:12.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 -- 172.21.15.92:0/3043628860 reap_dead start 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe37fe700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc005200 0x7fdfcc00ba10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c190 tx=0x7fdfcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfcc3cd83432250 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc001db0 con 0x7fdfcc005200 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc00ccf0 con 0x7fdfcc005200 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.584+0000 7fdfe1ffb700 1 -- 172.21.15.92:0/3043628860 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc003c40 con 0x7fdfcc005200 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.585+0000 7fdfe96f0700 1 -- 172.21.15.92:0/3043628860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc005200 msgr2=0x7fdfcc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.585+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc005200 0x7fdfcc00ba10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00c190 tx=0x7fdfcc005880 comp rx=0 tx=0).stop 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.585+0000 7fdfe96f0700 1 -- 172.21.15.92:0/3043628860 shutdown_connections 2023-06-09T04:36:12.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.585+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/3043628860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfcc005200 0x7fdfcc00ba10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.585+0000 7fdfe96f0700 1 -- 172.21.15.92:0/3043628860 >> 172.21.15.92:0/3043628860 conn(0x7fdfe406b2e0 msgr2=0x7fdfe406b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:12.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.585+0000 7fdfe96f0700 1 -- 172.21.15.92:0/3043628860 shutdown_connections 2023-06-09T04:36:12.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.585+0000 7fdfe96f0700 1 -- 172.21.15.92:0/3043628860 wait complete. 2023-06-09T04:36:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.586+0000 7fdfe96f0700 1 Processor -- start 2023-06-09T04:36:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.586+0000 7fdfe96f0700 1 -- start start 2023-06-09T04:36:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe96f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc005200 0x7fdfe40b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe96f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe40b6e40 0x7fdfe414c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe96f0700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfe41e6980 0x7fdfe41e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe96f0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40b8440 con 0x7fdfe40b6e40 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe96f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40b69b0 con 0x7fdfcc005200 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe96f0700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40b7e80 con 0x7fdfe41e6980 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe40b6e40 0x7fdfe414c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe40b6e40 0x7fdfe414c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39542/0 (socket says 172.21.15.92:39542) 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 -- 172.21.15.92:0/2294203704 learned_addr learned my addr 172.21.15.92:0/2294203704 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe2ffd700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc005200 0x7fdfe40b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe37fe700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfe41e6980 0x7fdfe41e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 -- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfe41e6980 msgr2=0x7fdfe41e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfe41e6980 0x7fdfe41e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 -- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc005200 msgr2=0x7fdfe40b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc005200 0x7fdfe40b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 -- 172.21.15.92:0/2294203704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe40b8160 con 0x7fdfe40b6e40 2023-06-09T04:36:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe2ffd700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc005200 0x7fdfe40b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:12.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.587+0000 7fdfe27fc700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe40b6e40 0x7fdfe414c4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40072c0 tx=0x7fdfd4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.588+0000 7fdfcb7fe700 1 -- 172.21.15.92:0/2294203704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40077a0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.588+0000 7fdfcb7fe700 1 -- 172.21.15.92:0/2294203704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd400f8d0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.588+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe40b6640 con 0x7fdfe40b6e40 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.588+0000 7fdfcb7fe700 1 -- 172.21.15.92:0/2294203704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400fc80 con 0x7fdfe40b6e40 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.588+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe40b72b0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.588+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe40b64a0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.589+0000 7fdfcb7fe700 1 -- 172.21.15.92:0/2294203704 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfd401d460 con 0x7fdfe40b6e40 2023-06-09T04:36:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.589+0000 7fdfcb7fe700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdfd006ace0 0x7fdfd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.589+0000 7fdfcb7fe700 1 -- 172.21.15.92:0/2294203704 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fdfd40893f0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.589+0000 7fdfe2ffd700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdfd006ace0 0x7fdfd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.590+0000 7fdfe2ffd700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdfd006ace0 0x7fdfd006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40b45d0 tx=0x7fdfd8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:12.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.594+0000 7fdfcb7fe700 1 -- 172.21.15.92:0/2294203704 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdfe40b64a0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.821+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdfe40b64a0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.826+0000 7fdfcb7fe700 1 -- 172.21.15.92:0/2294203704 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdfe40b64a0 con 0x7fdfe40b6e40 2023-06-09T04:36:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.832+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdfd006ace0 msgr2=0x7fdfd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.832+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdfd006ace0 0x7fdfd006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40b45d0 tx=0x7fdfd8008040 comp rx=0 tx=0).stop 2023-06-09T04:36:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.832+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe40b6e40 msgr2=0x7fdfe414c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.832+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe40b6e40 0x7fdfe414c4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40072c0 tx=0x7fdfd4007680 comp rx=0 tx=0).stop 2023-06-09T04:36:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.833+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 shutdown_connections 2023-06-09T04:36:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.833+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fdfd006ace0 0x7fdfd006d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.833+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfe41e6980 0x7fdfe41e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.833+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfcc005200 0x7fdfe40b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.833+0000 7fdfe96f0700 1 --2- 172.21.15.92:0/2294203704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdfe40b6e40 0x7fdfe414c4b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.833+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 >> 172.21.15.92:0/2294203704 conn(0x7fdfe406b2e0 msgr2=0x7fdfe40d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:12.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.837+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 shutdown_connections 2023-06-09T04:36:12.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.838+0000 7fdfe96f0700 1 -- 172.21.15.92:0/2294203704 wait complete. 2023-06-09T04:36:12.846 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 5 and data pool 6 2023-06-09T04:36:12.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:36:12.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.979+0000 7f099bd1a700 1 Processor -- start 2023-06-09T04:36:12.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.979+0000 7f099bd1a700 1 -- start start 2023-06-09T04:36:12.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f099bd1a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09940ac400 0x7f09940ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f099bd1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09940ce730 con 0x7f099415c7d0 2023-06-09T04:36:12.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f099bd1a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099406aaf0 con 0x7f09940cf010 2023-06-09T04:36:12.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f099bd1a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099415cba0 con 0x7f09940ac400 2023-06-09T04:36:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f0999ab6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f099415c7d0 0x7f09940ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37896/0 (socket says 172.21.15.92:37896) 2023-06-09T04:36:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f0999ab6700 1 -- 172.21.15.92:0/3140416447 learned_addr learned my addr 172.21.15.92:0/3140416447 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159749952 0 0) 0x7f0984001290 con 0x7f099415c7d0 2023-06-09T04:36:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09800035b0 con 0x7f099415c7d0 2023-06-09T04:36:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f099a2b7700 1 --2- 172.21.15.92:0/3140416447 >> v2:172.21.15.123:3300/0 conn(0x7f09940ac400 0x7f09940ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.980+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4245095797 0 0) 0x7f09800035b0 con 0x7f099415c7d0 2023-06-09T04:36:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 >> v2:172.21.15.123:3300/0 conn(0x7f09940ac400 msgr2=0x7f09940ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 --2- 172.21.15.92:0/3140416447 >> v2:172.21.15.123:3300/0 conn(0x7f09940ac400 0x7f09940ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 >> v1:172.21.15.123:6789/0 conn(0x7f09940cf010 legacy=0x7f099406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994069ee0 con 0x7f099415c7d0 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f0984001980 con 0x7f099415c7d0 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 >> v1:172.21.15.92:6789/0 conn(0x7f099415c7d0 legacy=0x7f09940ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0980003a00 0x7f0980003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980004850 0x7f0980004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0980005320 0x7f098000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099415cba0 con 0x7f0980005320 2023-06-09T04:36:12.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098000c1e0 con 0x7f0980003a00 2023-06-09T04:36:12.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098000c3a0 con 0x7f0980004850 2023-06-09T04:36:12.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 --2- 172.21.15.92:0/3140416447 >> v2:172.21.15.123:3300/0 conn(0x7f09940ac400 0x7f09940ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:12.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0980005320 0x7f098000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0999ab6700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0980003a00 0x7f0980003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f09992b5700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980004850 0x7f0980004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 -- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980004850 msgr2=0x7f0980004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980004850 0x7f0980004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 -- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0980003a00 msgr2=0x7f0980003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0980003a00 0x7f0980003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 -- 172.21.15.92:0/3140416447 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800035b0 con 0x7f0980005320 2023-06-09T04:36:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 -- 172.21.15.92:0/3140416447 reap_dead start 2023-06-09T04:36:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f099a2b7700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0980005320 0x7f098000ba10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f09940cd3d0 tx=0x7f0980005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fdd0437af31613 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c009b20 con 0x7f0980005320 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c003a20 con 0x7f0980005320 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.981+0000 7f0998ab4700 1 -- 172.21.15.92:0/3140416447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c003dd0 con 0x7f0980005320 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.982+0000 7f099bd1a700 1 -- 172.21.15.92:0/3140416447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0980005320 msgr2=0x7f098000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.982+0000 7f099bd1a700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0980005320 0x7f098000ba10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f09940cd3d0 tx=0x7f0980005880 comp rx=0 tx=0).stop 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.982+0000 7f099bd1a700 1 -- 172.21.15.92:0/3140416447 shutdown_connections 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.982+0000 7f099bd1a700 1 --2- 172.21.15.92:0/3140416447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0980005320 0x7f098000ba10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.982+0000 7f099bd1a700 1 -- 172.21.15.92:0/3140416447 >> 172.21.15.92:0/3140416447 conn(0x7f0994098c10 msgr2=0x7f09940cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:12.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.982+0000 7f099bd1a700 1 -- 172.21.15.92:0/3140416447 shutdown_connections 2023-06-09T04:36:12.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.982+0000 7f099bd1a700 1 -- 172.21.15.92:0/3140416447 wait complete. 2023-06-09T04:36:12.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.983+0000 7f099bd1a700 1 Processor -- start 2023-06-09T04:36:12.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.983+0000 7f099bd1a700 1 -- start start 2023-06-09T04:36:12.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099bd1a700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980005320 0x7f099415a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099bd1a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f099415ab50 0x7f09941e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099bd1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941e3560 0x7f09941599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099bd1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941e6de0 con 0x7f09941e3560 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099bd1a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941e70c0 con 0x7f099415ab50 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099bd1a700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09941e73a0 con 0x7f0980005320 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941e3560 0x7f09941599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941e3560 0x7f09941599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39580/0 (socket says 172.21.15.92:39580) 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 -- 172.21.15.92:0/1467895234 learned_addr learned my addr 172.21.15.92:0/1467895234 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f09992b5700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f099415ab50 0x7f09941e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f0999ab6700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980005320 0x7f099415a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 -- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980005320 msgr2=0x7f099415a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980005320 0x7f099415a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 -- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f099415ab50 msgr2=0x7f09941e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f099415ab50 0x7f09941e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.984+0000 7f099a2b7700 1 -- 172.21.15.92:0/1467895234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09940b3720 con 0x7f09941e3560 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.985+0000 7f099a2b7700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941e3560 0x7f09941599d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f099406c220 tx=0x7f098c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.985+0000 7f098a7fc700 1 -- 172.21.15.92:0/1467895234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c007000 con 0x7f09941e3560 2023-06-09T04:36:12.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.985+0000 7f098a7fc700 1 -- 172.21.15.92:0/1467895234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c004830 con 0x7f09941e3560 2023-06-09T04:36:12.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.985+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0994159f10 con 0x7f09941e3560 2023-06-09T04:36:12.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.985+0000 7f098a7fc700 1 -- 172.21.15.92:0/1467895234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c009b20 con 0x7f09941e3560 2023-06-09T04:36:12.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.985+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09940b3c40 con 0x7f09941e3560 2023-06-09T04:36:12.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.986+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09940b3960 con 0x7f09941e3560 2023-06-09T04:36:12.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.986+0000 7f098a7fc700 1 -- 172.21.15.92:0/1467895234 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f098c01f660 con 0x7f09941e3560 2023-06-09T04:36:12.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.986+0000 7f098a7fc700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f097806ace0 0x7f097806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:12.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.987+0000 7f0999ab6700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f097806ace0 0x7f097806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:12.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.987+0000 7f098a7fc700 1 -- 172.21.15.92:0/1467895234 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f098c08ada0 con 0x7f09941e3560 2023-06-09T04:36:12.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.987+0000 7f0999ab6700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f097806ace0 0x7f097806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f099415b720 tx=0x7f0984009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:12.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:12.991+0000 7f098a7fc700 1 -- 172.21.15.92:0/1467895234 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f09940b3960 con 0x7f09941e3560 2023-06-09T04:36:13.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.177+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09940b3960 con 0x7f09941e3560 2023-06-09T04:36:13.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.178+0000 7f098a7fc700 1 -- 172.21.15.92:0/1467895234 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18340 (secure 0 0 0) 0x7f09940b3960 con 0x7f09941e3560 2023-06-09T04:36:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:13.180 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:36:12.823460+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:36:11.118404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-09T04:36:11.791966+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:36:13.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f097806ace0 msgr2=0x7f097806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f097806ace0 0x7f097806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f099415b720 tx=0x7f0984009040 comp rx=0 tx=0).stop 2023-06-09T04:36:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941e3560 msgr2=0x7f09941599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941e3560 0x7f09941599d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f099406c220 tx=0x7f098c004b40 comp rx=0 tx=0).stop 2023-06-09T04:36:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 shutdown_connections 2023-06-09T04:36:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f097806ace0 0x7f097806d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09941e3560 0x7f09941599d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0980005320 0x7f099415a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 --2- 172.21.15.92:0/1467895234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f099415ab50 0x7f09941e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 >> 172.21.15.92:0/1467895234 conn(0x7f0994098c10 msgr2=0x7f09940afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:13.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 shutdown_connections 2023-06-09T04:36:13.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.180+0000 7f099bd1a700 1 -- 172.21.15.92:0/1467895234 wait complete. 2023-06-09T04:36:13.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:36:13.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.312+0000 7f87bf16d700 1 Processor -- start 2023-06-09T04:36:13.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.312+0000 7f87bf16d700 1 -- start start 2023-06-09T04:36:13.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87bf16d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87b815a1d0 0x7f87b80ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87bf16d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87b80cc0f0 0x7f87b8070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87bf16d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b80b1e90 con 0x7f87b815e800 2023-06-09T04:36:13.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87bf16d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b806b200 con 0x7f87b815a1d0 2023-06-09T04:36:13.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87bf16d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b80cd480 con 0x7f87b80cc0f0 2023-06-09T04:36:13.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87bcf09700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87b815a1d0 0x7f87b80ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87b7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f87b815e800 0x7f87b80cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37916/0 (socket says 172.21.15.92:37916) 2023-06-09T04:36:13.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.313+0000 7f87b7fff700 1 -- 172.21.15.92:0/2933113984 learned_addr learned my addr 172.21.15.92:0/2933113984 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:13.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bd70a700 1 --2- 172.21.15.92:0/2933113984 >> v2:172.21.15.123:3300/0 conn(0x7f87b80cc0f0 0x7f87b8070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bcf09700 1 -- 172.21.15.92:0/2933113984 >> v2:172.21.15.123:3300/0 conn(0x7f87b80cc0f0 msgr2=0x7f87b8070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bcf09700 1 --2- 172.21.15.92:0/2933113984 >> v2:172.21.15.123:3300/0 conn(0x7f87b80cc0f0 0x7f87b8070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bcf09700 1 -- 172.21.15.92:0/2933113984 >> v1:172.21.15.92:6789/0 conn(0x7f87b815e800 legacy=0x7f87b80cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bcf09700 1 -- 172.21.15.92:0/2933113984 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8066100 con 0x7f87b815a1d0 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87b77fe700 1 -- 172.21.15.92:0/2933113984 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091357742 0 0) 0x7f87b80b1e90 con 0x7f87b815e800 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bcf09700 1 --2- 172.21.15.92:0/2933113984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b815a1d0 0x7f87b80ccf40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87b8070a80 tx=0x7f87ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689d73464f81d7c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87b77fe700 1 -- 172.21.15.92:0/2933113984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac011830 con 0x7f87b815a1d0 2023-06-09T04:36:13.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87b77fe700 1 -- 172.21.15.92:0/2933113984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac011990 con 0x7f87b815a1d0 2023-06-09T04:36:13.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87b77fe700 1 -- 172.21.15.92:0/2933113984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac011d20 con 0x7f87b815a1d0 2023-06-09T04:36:13.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bf16d700 1 -- 172.21.15.92:0/2933113984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b815a1d0 msgr2=0x7f87b80ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bf16d700 1 --2- 172.21.15.92:0/2933113984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b815a1d0 0x7f87b80ccf40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87b8070a80 tx=0x7f87ac009b20 comp rx=0 tx=0).stop 2023-06-09T04:36:13.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bf16d700 1 -- 172.21.15.92:0/2933113984 shutdown_connections 2023-06-09T04:36:13.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bf16d700 1 --2- 172.21.15.92:0/2933113984 >> v2:172.21.15.123:3300/0 conn(0x7f87b80cc0f0 0x7f87b8070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.314+0000 7f87bf16d700 1 --2- 172.21.15.92:0/2933113984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b815a1d0 0x7f87b80ccf40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.315+0000 7f87bf16d700 1 -- 172.21.15.92:0/2933113984 >> 172.21.15.92:0/2933113984 conn(0x7f87b80b1800 msgr2=0x7f87b806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.315+0000 7f87bf16d700 1 -- 172.21.15.92:0/2933113984 shutdown_connections 2023-06-09T04:36:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.315+0000 7f87bf16d700 1 -- 172.21.15.92:0/2933113984 wait complete. 2023-06-09T04:36:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.316+0000 7f87bf16d700 1 Processor -- start 2023-06-09T04:36:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.316+0000 7f87bf16d700 1 -- start start 2023-06-09T04:36:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b80cc0f0 0x7f87b80c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87b815a1d0 0x7f87b815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87b815e800 0x7f87b815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80c6a60 con 0x7f87b80cc0f0 2023-06-09T04:36:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80ca380 con 0x7f87b815e800 2023-06-09T04:36:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80ca660 con 0x7f87b815a1d0 2023-06-09T04:36:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b80cc0f0 0x7f87b80c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b80cc0f0 0x7f87b80c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39590/0 (socket says 172.21.15.92:39590) 2023-06-09T04:36:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 -- 172.21.15.92:0/1868815339 learned_addr learned my addr 172.21.15.92:0/1868815339 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bd70a700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87b815e800 0x7f87b815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87b7fff700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87b815a1d0 0x7f87b815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 -- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87b815a1d0 msgr2=0x7f87b815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87b815a1d0 0x7f87b815cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 -- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87b815e800 msgr2=0x7f87b815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87b815e800 0x7f87b815d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 -- 172.21.15.92:0/1868815339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0094a0 con 0x7f87b80cc0f0 2023-06-09T04:36:13.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bcf09700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b80cc0f0 0x7f87b80c8d00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000c00 tx=0x7f87ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87b57fa700 1 -- 172.21.15.92:0/1868815339 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac004560 con 0x7f87b80cc0f0 2023-06-09T04:36:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87b57fa700 1 -- 172.21.15.92:0/1868815339 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac01f070 con 0x7f87b80cc0f0 2023-06-09T04:36:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b80ca940 con 0x7f87b80cc0f0 2023-06-09T04:36:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87b57fa700 1 -- 172.21.15.92:0/1868815339 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0038e0 con 0x7f87b80cc0f0 2023-06-09T04:36:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.317+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b80aced0 con 0x7f87b80cc0f0 2023-06-09T04:36:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.318+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8065e00 con 0x7f87b80cc0f0 2023-06-09T04:36:13.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.319+0000 7f87b57fa700 1 -- 172.21.15.92:0/1868815339 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87ac021030 con 0x7f87b80cc0f0 2023-06-09T04:36:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.319+0000 7f87b57fa700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f879c06ace0 0x7f879c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.319+0000 7f87b57fa700 1 -- 172.21.15.92:0/1868815339 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f87ac08a830 con 0x7f87b80cc0f0 2023-06-09T04:36:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.319+0000 7f87b7fff700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f879c06ace0 0x7f879c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.319+0000 7f87b7fff700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f879c06ace0 0x7f879c06d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87b8158ab0 tx=0x7f87a4007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:13.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.323+0000 7f87b57fa700 1 -- 172.21.15.92:0/1868815339 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f87b8065e00 con 0x7f87b80cc0f0 2023-06-09T04:36:13.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.512+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87b8065e00 con 0x7f87b80cc0f0 2023-06-09T04:36:13.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.514+0000 7f87b57fa700 1 -- 172.21.15.92:0/1868815339 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18340 (secure 0 0 0) 0x7f87b8065e00 con 0x7f87b80cc0f0 2023-06-09T04:36:13.515 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:13.516 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":37,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:36:12.823460+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:36:11.118404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-09T04:36:11.791966+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:36:13.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f879c06ace0 msgr2=0x7f879c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f879c06ace0 0x7f879c06d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87b8158ab0 tx=0x7f87a4007be0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b80cc0f0 msgr2=0x7f87b80c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b80cc0f0 0x7f87b80c8d00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000c00 tx=0x7f87ac004750 comp rx=0 tx=0).stop 2023-06-09T04:36:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 shutdown_connections 2023-06-09T04:36:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f879c06ace0 0x7f879c06d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87b815e800 0x7f87b815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87b80cc0f0 0x7f87b80c8d00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 --2- 172.21.15.92:0/1868815339 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87b815a1d0 0x7f87b815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 >> 172.21.15.92:0/1868815339 conn(0x7f87b80b1800 msgr2=0x7f87b80aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 shutdown_connections 2023-06-09T04:36:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.516+0000 7f87bf16d700 1 -- 172.21.15.92:0/1868815339 wait complete. 2023-06-09T04:36:13.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-09T04:36:13.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.650+0000 7f34f2ffa700 1 Processor -- start 2023-06-09T04:36:13.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f2ffa700 1 -- start start 2023-06-09T04:36:13.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f2ffa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f2ffa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f34ec0cf670 0x7f34ec0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f2ffa700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec0d0010 con 0x7f34ec15d2a0 2023-06-09T04:36:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f2ffa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec0d2b00 con 0x7f34ec0cf670 2023-06-09T04:36:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f2ffa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0d39d0 con 0x7f34ec15ced0 2023-06-09T04:36:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f1597700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f34ec0cf670 0x7f34ec0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.651+0000 7f34f1597700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f34ec0cf670 0x7f34ec0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39602/0 (socket says 172.21.15.92:39602) 2023-06-09T04:36:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34f1597700 1 -- 172.21.15.92:0/178173901 learned_addr learned my addr 172.21.15.92:0/178173901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34f0d96700 1 --2- 172.21.15.92:0/178173901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34f1597700 1 -- 172.21.15.92:0/178173901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 msgr2=0x7f34ec0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34f1597700 1 --2- 172.21.15.92:0/178173901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec0cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34f1597700 1 -- 172.21.15.92:0/178173901 >> v1:172.21.15.123:6789/0 conn(0x7f34ec15d2a0 legacy=0x7f34ec0d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34f1597700 1 -- 172.21.15.92:0/178173901 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0cd3c0 con 0x7f34ec0cf670 2023-06-09T04:36:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34f1597700 1 --2- 172.21.15.92:0/178173901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec0cf670 0x7f34ec0cfad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009e50 tx=0x7f34e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e976834b2174d4b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34eb7fe700 1 -- 172.21.15.92:0/178173901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400e070 con 0x7f34ec0cf670 2023-06-09T04:36:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34eb7fe700 1 -- 172.21.15.92:0/178173901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e400b9c0 con 0x7f34ec0cf670 2023-06-09T04:36:13.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.652+0000 7f34eb7fe700 1 -- 172.21.15.92:0/178173901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400bd70 con 0x7f34ec0cf670 2023-06-09T04:36:13.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 -- 172.21.15.92:0/178173901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec0cf670 msgr2=0x7f34ec0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/178173901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec0cf670 0x7f34ec0cfad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009e50 tx=0x7f34e400b330 comp rx=0 tx=0).stop 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 -- 172.21.15.92:0/178173901 shutdown_connections 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/178173901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec0cf670 0x7f34ec0cfad0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/178173901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec0cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 -- 172.21.15.92:0/178173901 >> 172.21.15.92:0/178173901 conn(0x7f34ec06b2e0 msgr2=0x7f34ec06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 -- 172.21.15.92:0/178173901 shutdown_connections 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.653+0000 7f34f2ffa700 1 -- 172.21.15.92:0/178173901 wait complete. 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.654+0000 7f34f2ffa700 1 Processor -- start 2023-06-09T04:36:13.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.654+0000 7f34f2ffa700 1 -- start start 2023-06-09T04:36:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f2ffa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34ec0cf670 0x7f34ec0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f2ffa700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec1e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f2ffa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec15d2a0 0x7f34ec14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f2ffa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1e65b0 con 0x7f34ec15d2a0 2023-06-09T04:36:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f2ffa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1e6890 con 0x7f34ec0cf670 2023-06-09T04:36:13.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f2ffa700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1e6b70 con 0x7f34ec15ced0 2023-06-09T04:36:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec15d2a0 0x7f34ec14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec15d2a0 0x7f34ec14ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39608/0 (socket says 172.21.15.92:39608) 2023-06-09T04:36:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 -- 172.21.15.92:0/201486246 learned_addr learned my addr 172.21.15.92:0/201486246 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f0d96700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34ec0cf670 0x7f34ec0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34ebfff700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec1e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 -- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 msgr2=0x7f34ec1e5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec1e5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 -- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34ec0cf670 msgr2=0x7f34ec0c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34ec0cf670 0x7f34ec0c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 -- 172.21.15.92:0/201486246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4009b30 con 0x7f34ec15d2a0 2023-06-09T04:36:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f0d96700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34ec0cf670 0x7f34ec0c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f1597700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec15d2a0 0x7f34ec14ac30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f34ec06bfa0 tx=0x7f34e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34e97fa700 1 -- 172.21.15.92:0/201486246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4004960 con 0x7f34ec15d2a0 2023-06-09T04:36:13.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34e97fa700 1 -- 172.21.15.92:0/201486246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e40099b0 con 0x7f34ec15d2a0 2023-06-09T04:36:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34e97fa700 1 -- 172.21.15.92:0/201486246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e401b670 con 0x7f34ec15d2a0 2023-06-09T04:36:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.655+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec1e6e50 con 0x7f34ec15d2a0 2023-06-09T04:36:13.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.656+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec1e62a0 con 0x7f34ec15d2a0 2023-06-09T04:36:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.656+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec04e8e0 con 0x7f34ec15d2a0 2023-06-09T04:36:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.657+0000 7f34e97fa700 1 -- 172.21.15.92:0/201486246 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34e401b7d0 con 0x7f34ec15d2a0 2023-06-09T04:36:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.657+0000 7f34e97fa700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f34d006ace0 0x7f34d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.657+0000 7f34e97fa700 1 -- 172.21.15.92:0/201486246 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f34e408adb0 con 0x7f34ec15d2a0 2023-06-09T04:36:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.657+0000 7f34f0d96700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f34d006ace0 0x7f34d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.658+0000 7f34f0d96700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f34d006ace0 0x7f34d006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f34ec15eaf0 tx=0x7f34e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:13.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.662+0000 7f34e97fa700 1 -- 172.21.15.92:0/201486246 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f34e4055ef0 con 0x7f34ec15d2a0 2023-06-09T04:36:13.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:13.894+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f34ec0b50c0 con 0x7f34ec15d2a0 2023-06-09T04:36:14.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.838+0000 7f34e97fa700 1 -- 172.21.15.92:0/201486246 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f34e4059510 con 0x7f34ec15d2a0 2023-06-09T04:36:14.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f34d006ace0 msgr2=0x7f34d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f34d006ace0 0x7f34d006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f34ec15eaf0 tx=0x7f34e0008040 comp rx=0 tx=0).stop 2023-06-09T04:36:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec15d2a0 msgr2=0x7f34ec14ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec15d2a0 0x7f34ec14ac30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f34ec06bfa0 tx=0x7f34e4004ad0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 shutdown_connections 2023-06-09T04:36:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f34d006ace0 0x7f34d006d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec15d2a0 0x7f34ec14ac30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34ec0cf670 0x7f34ec0c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 --2- 172.21.15.92:0/201486246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34ec15ced0 0x7f34ec1e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 >> 172.21.15.92:0/201486246 conn(0x7f34ec06b2e0 msgr2=0x7f34ec0d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:14.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 shutdown_connections 2023-06-09T04:36:14.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.841+0000 7f34f2ffa700 1 -- 172.21.15.92:0/201486246 wait complete. 2023-06-09T04:36:14.857 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-09T04:36:14.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-09T04:36:14.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.985+0000 7fd127983700 1 Processor -- start 2023-06-09T04:36:14.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd127983700 1 -- start start 2023-06-09T04:36:14.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd127983700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd12006cbb0 0x7fd12006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:14.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd127983700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 0x7fd120069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:14.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd127983700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12006d4c0 con 0x7fd12006d8f0 2023-06-09T04:36:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd127983700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200cdcb0 con 0x7fd12006cbb0 2023-06-09T04:36:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd127983700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200cdf90 con 0x7fd1201596c0 2023-06-09T04:36:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd12571f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd12006cbb0 0x7fd12006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd12571f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd12006cbb0 0x7fd12006cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39612/0 (socket says 172.21.15.92:39612) 2023-06-09T04:36:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.986+0000 7fd12571f700 1 -- 172.21.15.92:0/3174617183 learned_addr learned my addr 172.21.15.92:0/3174617183 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd12571f700 1 -- 172.21.15.92:0/3174617183 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 msgr2=0x7fd120069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:14.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd12571f700 1 --2- 172.21.15.92:0/3174617183 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 0x7fd120069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd12571f700 1 -- 172.21.15.92:0/3174617183 >> v1:172.21.15.123:6789/0 conn(0x7fd12006d8f0 legacy=0x7fd120159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:14.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd12571f700 1 -- 172.21.15.92:0/3174617183 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1200ce270 con 0x7fd12006cbb0 2023-06-09T04:36:14.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd12571f700 1 --2- 172.21.15.92:0/3174617183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd12006cf80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd1100098f0 tx=0x7fd110009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4619338928d3e0c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:14.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd117fff700 1 -- 172.21.15.92:0/3174617183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110004230 con 0x7fd12006cbb0 2023-06-09T04:36:14.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd117fff700 1 -- 172.21.15.92:0/3174617183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd110004390 con 0x7fd12006cbb0 2023-06-09T04:36:14.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd117fff700 1 -- 172.21.15.92:0/3174617183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1100036a0 con 0x7fd12006cbb0 2023-06-09T04:36:14.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd127983700 1 -- 172.21.15.92:0/3174617183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 msgr2=0x7fd12006cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.987+0000 7fd127983700 1 --2- 172.21.15.92:0/3174617183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd12006cf80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd1100098f0 tx=0x7fd110009c00 comp rx=0 tx=0).stop 2023-06-09T04:36:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.988+0000 7fd127983700 1 -- 172.21.15.92:0/3174617183 shutdown_connections 2023-06-09T04:36:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.988+0000 7fd127983700 1 --2- 172.21.15.92:0/3174617183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd12006cf80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.988+0000 7fd127983700 1 --2- 172.21.15.92:0/3174617183 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 0x7fd120069db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.988+0000 7fd127983700 1 -- 172.21.15.92:0/3174617183 >> 172.21.15.92:0/3174617183 conn(0x7fd1200cfc40 msgr2=0x7fd1200b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.988+0000 7fd127983700 1 -- 172.21.15.92:0/3174617183 shutdown_connections 2023-06-09T04:36:14.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.988+0000 7fd127983700 1 -- 172.21.15.92:0/3174617183 wait complete. 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.989+0000 7fd127983700 1 Processor -- start 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.989+0000 7fd127983700 1 -- start start 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.989+0000 7fd127983700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd120145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.989+0000 7fd127983700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd12006d8f0 0x7fd1200c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.989+0000 7fd127983700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 0x7fd1200c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd127983700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1201459a0 con 0x7fd12006cbb0 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd127983700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200c73f0 con 0x7fd12006d8f0 2023-06-09T04:36:14.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd127983700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200c76d0 con 0x7fd1201596c0 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd120145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd120145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39618/0 (socket says 172.21.15.92:39618) 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 -- 172.21.15.92:0/215694317 learned_addr learned my addr 172.21.15.92:0/215694317 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd124f1e700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd12006d8f0 0x7fd1200c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd125f20700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 0x7fd1200c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 -- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 msgr2=0x7fd1200c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 0x7fd1200c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 -- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd12006d8f0 msgr2=0x7fd1200c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:14.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd12006d8f0 0x7fd1200c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:14.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 -- 172.21.15.92:0/215694317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd110009550 con 0x7fd12006cbb0 2023-06-09T04:36:14.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd12571f700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd120145460 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd110009130 tx=0x7fd110004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:14.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd115ffb700 1 -- 172.21.15.92:0/215694317 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11001f070 con 0x7fd12006cbb0 2023-06-09T04:36:14.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd115ffb700 1 -- 172.21.15.92:0/215694317 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd110003ca0 con 0x7fd12006cbb0 2023-06-09T04:36:14.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd115ffb700 1 -- 172.21.15.92:0/215694317 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000b8a0 con 0x7fd12006cbb0 2023-06-09T04:36:14.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.990+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1200c79b0 con 0x7fd12006cbb0 2023-06-09T04:36:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.991+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1200c6c10 con 0x7fd12006cbb0 2023-06-09T04:36:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.991+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12004e8e0 con 0x7fd12006cbb0 2023-06-09T04:36:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.992+0000 7fd115ffb700 1 -- 172.21.15.92:0/215694317 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd11000ba00 con 0x7fd12006cbb0 2023-06-09T04:36:14.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.992+0000 7fd115ffb700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd10406ace0 0x7fd10406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.992+0000 7fd115ffb700 1 -- 172.21.15.92:0/215694317 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd11008c3e0 con 0x7fd12006cbb0 2023-06-09T04:36:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.992+0000 7fd124f1e700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd10406ace0 0x7fd10406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.993+0000 7fd124f1e700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd10406ace0 0x7fd10406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1200ca6e0 tx=0x7fd11c009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:14.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:14.997+0000 7fd115ffb700 1 -- 172.21.15.92:0/215694317 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd110057520 con 0x7fd12006cbb0 2023-06-09T04:36:15.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.230+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd1200c5e80 con 0x7fd12006cbb0 2023-06-09T04:36:15.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.840+0000 7fd115ffb700 1 -- 172.21.15.92:0/215694317 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd11005ab40 con 0x7fd12006cbb0 2023-06-09T04:36:15.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.843+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd10406ace0 msgr2=0x7fd10406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:15.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.843+0000 7fd127983700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd10406ace0 0x7fd10406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd1200ca6e0 tx=0x7fd11c009110 comp rx=0 tx=0).stop 2023-06-09T04:36:15.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.843+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 msgr2=0x7fd120145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:15.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.843+0000 7fd127983700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd120145460 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd110009130 tx=0x7fd110004750 comp rx=0 tx=0).stop 2023-06-09T04:36:15.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 shutdown_connections 2023-06-09T04:36:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fd10406ace0 0x7fd10406d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1201596c0 0x7fd1200c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd12006cbb0 0x7fd120145460 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 --2- 172.21.15.92:0/215694317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd12006d8f0 0x7fd1200c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 >> 172.21.15.92:0/215694317 conn(0x7fd1200cfc40 msgr2=0x7fd1200ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 shutdown_connections 2023-06-09T04:36:15.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.844+0000 7fd127983700 1 -- 172.21.15.92:0/215694317 wait complete. 2023-06-09T04:36:15.859 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-09T04:36:15.860 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:36:15.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.984+0000 7fa7c3838700 1 Processor -- start 2023-06-09T04:36:15.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.985+0000 7fa7c3838700 1 -- start start 2023-06-09T04:36:15.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.985+0000 7fa7c3838700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7bc0ac400 0x7fa7bc0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:15.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.985+0000 7fa7c3838700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0ce730 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.985+0000 7fa7c3838700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc06aaf0 con 0x7fa7bc0cf010 2023-06-09T04:36:15.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.985+0000 7fa7c3838700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc15cba0 con 0x7fa7bc0ac400 2023-06-09T04:36:15.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7c1dd5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7bc0ac400 0x7fa7bc0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:15.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7c15d4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa7bc15c7d0 0x7fa7bc0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37956/0 (socket says 172.21.15.92:37956) 2023-06-09T04:36:15.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7c15d4700 1 -- 172.21.15.92:0/4250439224 learned_addr learned my addr 172.21.15.92:0/4250439224 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:15.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7c1dd5700 1 -- 172.21.15.92:0/4250439224 >> v1:172.21.15.123:6789/0 conn(0x7fa7bc0cf010 legacy=0x7fa7bc06e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:15.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7c1dd5700 1 -- 172.21.15.92:0/4250439224 >> v1:172.21.15.92:6789/0 conn(0x7fa7bc15c7d0 legacy=0x7fa7bc0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:15.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7b3fff700 1 -- 172.21.15.92:0/4250439224 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724308681 0 0) 0x7fa7ac001290 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7c1dd5700 1 -- 172.21.15.92:0/4250439224 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc069ee0 con 0x7fa7bc0ac400 2023-06-09T04:36:15.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7c1dd5700 1 --2- 172.21.15.92:0/4250439224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0ac400 0x7fa7bc0ce1f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009d90 tx=0x7fa7b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca0da02b61003e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:15.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7b3fff700 1 -- 172.21.15.92:0/4250439224 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400e070 con 0x7fa7bc0ac400 2023-06-09T04:36:15.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7b3fff700 1 -- 172.21.15.92:0/4250439224 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b400bbd0 con 0x7fa7bc0ac400 2023-06-09T04:36:15.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.986+0000 7fa7b3fff700 1 -- 172.21.15.92:0/4250439224 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4012430 con 0x7fa7bc0ac400 2023-06-09T04:36:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.987+0000 7fa7c3838700 1 -- 172.21.15.92:0/4250439224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0ac400 msgr2=0x7fa7bc0ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.987+0000 7fa7c3838700 1 --2- 172.21.15.92:0/4250439224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0ac400 0x7fa7bc0ce1f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009d90 tx=0x7fa7b400b330 comp rx=0 tx=0).stop 2023-06-09T04:36:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.987+0000 7fa7c3838700 1 -- 172.21.15.92:0/4250439224 shutdown_connections 2023-06-09T04:36:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.987+0000 7fa7c3838700 1 --2- 172.21.15.92:0/4250439224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc0ac400 0x7fa7bc0ce1f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.987+0000 7fa7c3838700 1 -- 172.21.15.92:0/4250439224 >> 172.21.15.92:0/4250439224 conn(0x7fa7bc098c10 msgr2=0x7fa7bc0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.987+0000 7fa7c3838700 1 -- 172.21.15.92:0/4250439224 shutdown_connections 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.987+0000 7fa7c3838700 1 -- 172.21.15.92:0/4250439224 wait complete. 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.988+0000 7fa7c3838700 1 Processor -- start 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.988+0000 7fa7c3838700 1 -- start start 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c3838700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7bc0ac400 0x7fa7bc14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c3838700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c3838700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc15c7d0 0x7fa7bc0cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c3838700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc14c2f0 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c3838700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c40b0 con 0x7fa7bc0ac400 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c3838700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c4390 con 0x7fa7bc0cf010 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc15c7d0 0x7fa7bc0cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc15c7d0 0x7fa7bc0cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39634/0 (socket says 172.21.15.92:39634) 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 -- 172.21.15.92:0/359264891 learned_addr learned my addr 172.21.15.92:0/359264891 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c15d4700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7bc0ac400 0x7fa7bc14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c0dd3700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 -- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 msgr2=0x7fa7bc0cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:15.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 -- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7bc0ac400 msgr2=0x7fa7bc14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7bc0ac400 0x7fa7bc14bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 -- 172.21.15.92:0/359264891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4009a70 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c0dd3700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c1dd5700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc15c7d0 0x7fa7bc0cc2a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4014040 tx=0x7fa7b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7b1ffb700 1 -- 172.21.15.92:0/359264891 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4012430 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7b1ffb700 1 -- 172.21.15.92:0/359264891 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b4020070 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc0c4670 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.989+0000 7fa7b1ffb700 1 -- 172.21.15.92:0/359264891 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4004280 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.990+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc0c4c30 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.991+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc0740e0 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.991+0000 7fa7b1ffb700 1 -- 172.21.15.92:0/359264891 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7b40043e0 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.991+0000 7fa7b1ffb700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa7a006ace0 0x7fa7a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.991+0000 7fa7b1ffb700 1 -- 172.21.15.92:0/359264891 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7b4022070 con 0x7fa7bc15c7d0 2023-06-09T04:36:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.991+0000 7fa7c15d4700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa7a006ace0 0x7fa7a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.992+0000 7fa7c15d4700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa7a006ace0 0x7fa7a006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac007ae0 tx=0x7fa7ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:15.996+0000 7fa7b1ffb700 1 -- 172.21.15.92:0/359264891 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa7b40565e0 con 0x7fa7bc15c7d0 2023-06-09T04:36:16.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:36:16.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:36:16.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.233+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7bc0cd1b0 con 0x7fa7bc15c7d0 2023-06-09T04:36:16.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.233+0000 7fa7b1ffb700 1 -- 172.21.15.92:0/359264891 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fa7b4059c00 con 0x7fa7bc15c7d0 2023-06-09T04:36:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:16.235 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/2062617587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2062617587},{"type":"v1","addr":"172.21.15.123:6834","nonce":2062617587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1352099741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":1352099741},{"type":"v1","addr":"172.21.15.123:6835","nonce":1352099741}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2293053570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2293053570},{"type":"v1","addr":"172.21.15.92:6836","nonce":2293053570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:36:12.823788+0000","modified":"2023-06-09T04:36:15.838310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/637518437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":637518437},{"type":"v1","addr":"172.21.15.92:6837","nonce":637518437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-09T04:36:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa7a006ace0 msgr2=0x7fa7a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa7a006ace0 0x7fa7a006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac007ae0 tx=0x7fa7ac009040 comp rx=0 tx=0).stop 2023-06-09T04:36:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc15c7d0 msgr2=0x7fa7bc0cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc15c7d0 0x7fa7bc0cc2a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4014040 tx=0x7fa7b4004ad0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 shutdown_connections 2023-06-09T04:36:16.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fa7a006ace0 0x7fa7a006d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7bc15c7d0 0x7fa7bc0cc2a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7bc0ac400 0x7fa7bc14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 --2- 172.21.15.92:0/359264891 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7bc0cf010 0x7fa7bc0cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 >> 172.21.15.92:0/359264891 conn(0x7fa7bc098c10 msgr2=0x7fa7bc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:16.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 shutdown_connections 2023-06-09T04:36:16.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.235+0000 7fa7c3838700 1 -- 172.21.15.92:0/359264891 wait complete. 2023-06-09T04:36:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2023-06-09T04:36:16.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:36:16.274 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-06-09T04:36:16.273+0000 7f5702ffa700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T04:36:16.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.368+0000 7efef1981700 1 Processor -- start 2023-06-09T04:36:16.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.368+0000 7efef1981700 1 -- start start 2023-06-09T04:36:16.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.368+0000 7efef1981700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efeec15a380 0x7efeec0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.368+0000 7efef1981700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efeec0afef0 0x7efeec0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efef1981700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec0cdd10 con 0x7efeec0af9a0 2023-06-09T04:36:16.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efef1981700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec06eca0 con 0x7efeec15a380 2023-06-09T04:36:16.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efef1981700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec0d07f0 con 0x7efeec0afef0 2023-06-09T04:36:16.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeaffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efeec15a380 0x7efeec0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efeec0af9a0 0x7efeec06f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37972/0 (socket says 172.21.15.92:37972) 2023-06-09T04:36:16.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeb7fe700 1 -- 172.21.15.92:0/2625768429 learned_addr learned my addr 172.21.15.92:0/2625768429 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:16.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeea7fc700 1 --2- 172.21.15.92:0/2625768429 >> v2:172.21.15.123:3300/0 conn(0x7efeec0afef0 0x7efeec0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeaffd700 1 -- 172.21.15.92:0/2625768429 >> v2:172.21.15.123:3300/0 conn(0x7efeec0afef0 msgr2=0x7efeec0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeaffd700 1 --2- 172.21.15.92:0/2625768429 >> v2:172.21.15.123:3300/0 conn(0x7efeec0afef0 0x7efeec0cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeaffd700 1 -- 172.21.15.92:0/2625768429 >> v1:172.21.15.92:6789/0 conn(0x7efeec0af9a0 legacy=0x7efeec06f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeaffd700 1 -- 172.21.15.92:0/2625768429 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec15a750 con 0x7efeec15a380 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efee9ffb700 1 -- 172.21.15.92:0/2625768429 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729236473 0 0) 0x7efeec0cdd10 con 0x7efeec0af9a0 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efeeaffd700 1 --2- 172.21.15.92:0/2625768429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec15a380 0x7efeec0ce540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efeec071fd0 tx=0x7efedc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9b3e72d5d5974c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efee9ffb700 1 -- 172.21.15.92:0/2625768429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc011830 con 0x7efeec15a380 2023-06-09T04:36:16.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efee9ffb700 1 -- 172.21.15.92:0/2625768429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc011990 con 0x7efeec15a380 2023-06-09T04:36:16.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.369+0000 7efee9ffb700 1 -- 172.21.15.92:0/2625768429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc011d20 con 0x7efeec15a380 2023-06-09T04:36:16.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 -- 172.21.15.92:0/2625768429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec15a380 msgr2=0x7efeec0ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 --2- 172.21.15.92:0/2625768429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec15a380 0x7efeec0ce540 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efeec071fd0 tx=0x7efedc009b20 comp rx=0 tx=0).stop 2023-06-09T04:36:16.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 -- 172.21.15.92:0/2625768429 shutdown_connections 2023-06-09T04:36:16.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 --2- 172.21.15.92:0/2625768429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec15a380 0x7efeec0ce540 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 --2- 172.21.15.92:0/2625768429 >> v2:172.21.15.123:3300/0 conn(0x7efeec0afef0 0x7efeec0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 -- 172.21.15.92:0/2625768429 >> 172.21.15.92:0/2625768429 conn(0x7efeec089fe0 msgr2=0x7efeec0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:16.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 -- 172.21.15.92:0/2625768429 shutdown_connections 2023-06-09T04:36:16.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.370+0000 7efef1981700 1 -- 172.21.15.92:0/2625768429 wait complete. 2023-06-09T04:36:16.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.371+0000 7efef1981700 1 Processor -- start 2023-06-09T04:36:16.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.371+0000 7efef1981700 1 -- start start 2023-06-09T04:36:16.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efef1981700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0af9a0 0x7efeec0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efef1981700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec0afef0 0x7efeec153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efef1981700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec15a380 0x7efeec0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efef1981700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec09dba0 con 0x7efeec0afef0 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efef1981700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0c7690 con 0x7efeec15a380 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efef1981700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec09b3b0 con 0x7efeec0af9a0 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec0afef0 0x7efeec153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec0afef0 0x7efeec153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39646/0 (socket says 172.21.15.92:39646) 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 -- 172.21.15.92:0/3681659015 learned_addr learned my addr 172.21.15.92:0/3681659015 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeeb7fe700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec15a380 0x7efeec0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeeaffd700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0af9a0 0x7efeec0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 -- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0af9a0 msgr2=0x7efeec0c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0af9a0 0x7efeec0c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 -- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec15a380 msgr2=0x7efeec0b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec15a380 0x7efeec0b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 -- 172.21.15.92:0/3681659015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0094a0 con 0x7efeec0afef0 2023-06-09T04:36:16.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeeb7fe700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec15a380 0x7efeec0b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:16.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeeaffd700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0af9a0 0x7efeec0c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:16.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.372+0000 7efeea7fc700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec0afef0 0x7efeec153510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efee000c7f0 tx=0x7efee000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.373+0000 7efed3fff700 1 -- 172.21.15.92:0/3681659015 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0011800 con 0x7efeec0afef0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.373+0000 7efed3fff700 1 -- 172.21.15.92:0/3681659015 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0011960 con 0x7efeec0afef0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.373+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec09e690 con 0x7efeec0afef0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.373+0000 7efed3fff700 1 -- 172.21.15.92:0/3681659015 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0011cf0 con 0x7efeec0afef0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.373+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec09e0d0 con 0x7efeec0afef0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.374+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec0cedb0 con 0x7efeec0afef0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.374+0000 7efed3fff700 1 -- 172.21.15.92:0/3681659015 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efee0018360 con 0x7efeec0afef0 2023-06-09T04:36:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.374+0000 7efed3fff700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efed806ace0 0x7efed806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.374+0000 7efed3fff700 1 -- 172.21.15.92:0/3681659015 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efee0089460 con 0x7efeec0afef0 2023-06-09T04:36:16.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.374+0000 7efeeaffd700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efed806ace0 0x7efed806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.375+0000 7efeeaffd700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efed806ace0 0x7efed806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efedc0090b0 tx=0x7efedc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:16.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.379+0000 7efed3fff700 1 -- 172.21.15.92:0/3681659015 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efee0054510 con 0x7efeec0afef0 2023-06-09T04:36:16.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.567+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeec0740e0 con 0x7efeec0afef0 2023-06-09T04:36:16.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.569+0000 7efed3fff700 1 -- 172.21.15.92:0/3681659015 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18327 (secure 0 0 0) 0x7efee0057b30 con 0x7efeec0afef0 2023-06-09T04:36:16.570 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:16.570 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":38,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:36:13.825643+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:36:11.118404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-09T04:36:11.791966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:36:16.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efed806ace0 msgr2=0x7efed806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efed806ace0 0x7efed806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efedc0090b0 tx=0x7efedc008030 comp rx=0 tx=0).stop 2023-06-09T04:36:16.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec0afef0 msgr2=0x7efeec153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec0afef0 0x7efeec153510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efee000c7f0 tx=0x7efee000cbb0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 shutdown_connections 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7efed806ace0 0x7efed806d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeec15a380 0x7efeec0b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeec0af9a0 0x7efeec0c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 --2- 172.21.15.92:0/3681659015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeec0afef0 0x7efeec153510 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 >> 172.21.15.92:0/3681659015 conn(0x7efeec089fe0 msgr2=0x7efeec0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:16.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 shutdown_connections 2023-06-09T04:36:16.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.571+0000 7efef1981700 1 -- 172.21.15.92:0/3681659015 wait complete. 2023-06-09T04:36:16.584 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-09T04:36:16.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T04:36:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.742+0000 7f5a8e556700 1 Processor -- start 2023-06-09T04:36:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.742+0000 7f5a8e556700 1 -- start start 2023-06-09T04:36:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a8e556700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5a88158920 0x7f5a8815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a8e556700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a88158cf0 0x7f5a880cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a8e556700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8815d4a0 con 0x7f5a8806d220 2023-06-09T04:36:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a8e556700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8806d7c0 con 0x7f5a88158cf0 2023-06-09T04:36:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a8e556700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a88159f30 con 0x7f5a88158920 2023-06-09T04:36:16.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a88158cf0 0x7f5a880cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a8caf3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5a8806d220 0x7f5a88069db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37988/0 (socket says 172.21.15.92:37988) 2023-06-09T04:36:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a8caf3700 1 -- 172.21.15.92:0/1454934511 learned_addr learned my addr 172.21.15.92:0/1454934511 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a87fff700 1 --2- 172.21.15.92:0/1454934511 >> v2:172.21.15.123:3300/0 conn(0x7f5a88158920 0x7f5a8815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a877fe700 1 -- 172.21.15.92:0/1454934511 >> v2:172.21.15.123:3300/0 conn(0x7f5a88158920 msgr2=0x7f5a8815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a877fe700 1 --2- 172.21.15.92:0/1454934511 >> v2:172.21.15.123:3300/0 conn(0x7f5a88158920 0x7f5a8815cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a877fe700 1 -- 172.21.15.92:0/1454934511 >> v1:172.21.15.92:6789/0 conn(0x7f5a8806d220 legacy=0x7f5a88069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a877fe700 1 -- 172.21.15.92:0/1454934511 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a880cb540 con 0x7f5a88158cf0 2023-06-09T04:36:16.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.743+0000 7f5a86ffd700 1 -- 172.21.15.92:0/1454934511 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133284161 0 0) 0x7f5a8815d4a0 con 0x7f5a8806d220 2023-06-09T04:36:16.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a877fe700 1 --2- 172.21.15.92:0/1454934511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a88158cf0 0x7f5a880cd1e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5a88070400 tx=0x7f5a7c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2f219f21d6c56f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:16.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a86ffd700 1 -- 172.21.15.92:0/1454934511 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c011830 con 0x7f5a88158cf0 2023-06-09T04:36:16.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a86ffd700 1 -- 172.21.15.92:0/1454934511 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c011990 con 0x7f5a88158cf0 2023-06-09T04:36:16.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a86ffd700 1 -- 172.21.15.92:0/1454934511 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c011d20 con 0x7f5a88158cf0 2023-06-09T04:36:16.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 -- 172.21.15.92:0/1454934511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a88158cf0 msgr2=0x7f5a880cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 --2- 172.21.15.92:0/1454934511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a88158cf0 0x7f5a880cd1e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5a88070400 tx=0x7f5a7c009b20 comp rx=0 tx=0).stop 2023-06-09T04:36:16.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 -- 172.21.15.92:0/1454934511 shutdown_connections 2023-06-09T04:36:16.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 --2- 172.21.15.92:0/1454934511 >> v2:172.21.15.123:3300/0 conn(0x7f5a88158920 0x7f5a8815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 --2- 172.21.15.92:0/1454934511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a88158cf0 0x7f5a880cd1e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 -- 172.21.15.92:0/1454934511 >> 172.21.15.92:0/1454934511 conn(0x7f5a880b1800 msgr2=0x7f5a8806cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 -- 172.21.15.92:0/1454934511 shutdown_connections 2023-06-09T04:36:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.744+0000 7f5a8e556700 1 -- 172.21.15.92:0/1454934511 wait complete. 2023-06-09T04:36:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.745+0000 7f5a8e556700 1 Processor -- start 2023-06-09T04:36:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.746+0000 7f5a8e556700 1 -- start start 2023-06-09T04:36:16.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.746+0000 7f5a8e556700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a8806d220 0x7f5a88155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.746+0000 7f5a8e556700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a88158920 0x7f5a88149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.746+0000 7f5a8e556700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88151b30 0x7f5a88157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.746+0000 7f5a8e556700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a880acd90 con 0x7f5a8806d220 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.746+0000 7f5a8e556700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a880aca80 con 0x7f5a88151b30 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.746+0000 7f5a8e556700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88152580 con 0x7f5a88158920 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a8806d220 0x7f5a88155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a8806d220 0x7f5a88155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39658/0 (socket says 172.21.15.92:39658) 2023-06-09T04:36:16.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 -- 172.21.15.92:0/4242484642 learned_addr learned my addr 172.21.15.92:0/4242484642 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a8caf3700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88151b30 0x7f5a88157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a877fe700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a88158920 0x7f5a88149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 -- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a88158920 msgr2=0x7f5a88149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a88158920 0x7f5a88149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 -- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88151b30 msgr2=0x7f5a88157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88151b30 0x7f5a88157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 -- 172.21.15.92:0/4242484642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0094a0 con 0x7f5a8806d220 2023-06-09T04:36:16.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a87fff700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a8806d220 0x7f5a88155f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800e840 tx=0x7f5a7800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a84ff9700 1 -- 172.21.15.92:0/4242484642 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78013610 con 0x7f5a8806d220 2023-06-09T04:36:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a84ff9700 1 -- 172.21.15.92:0/4242484642 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a78013770 con 0x7f5a8806d220 2023-06-09T04:36:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a84ff9700 1 -- 172.21.15.92:0/4242484642 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78013b20 con 0x7f5a8806d220 2023-06-09T04:36:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a88152860 con 0x7f5a8806d220 2023-06-09T04:36:16.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.747+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a88149c50 con 0x7f5a8806d220 2023-06-09T04:36:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.748+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a880acbc0 con 0x7f5a8806d220 2023-06-09T04:36:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.748+0000 7f5a84ff9700 1 -- 172.21.15.92:0/4242484642 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a78013c80 con 0x7f5a8806d220 2023-06-09T04:36:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.749+0000 7f5a84ff9700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a7406ace0 0x7f5a7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.749+0000 7f5a84ff9700 1 -- 172.21.15.92:0/4242484642 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a78089140 con 0x7f5a8806d220 2023-06-09T04:36:16.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.749+0000 7f5a877fe700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a7406ace0 0x7f5a7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.749+0000 7f5a877fe700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a7406ace0 0x7f5a7406d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0090b0 tx=0x7f5a7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:16.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.753+0000 7f5a84ff9700 1 -- 172.21.15.92:0/4242484642 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a78057c90 con 0x7f5a8806d220 2023-06-09T04:36:16.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.940+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 --> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a88159450 con 0x7f5a7406ace0 2023-06-09T04:36:16.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.949+0000 7f5a84ff9700 1 -- 172.21.15.92:0/4242484642 <== mgr.4104 v2:172.21.15.92:6832/3112104013 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311412 (secure 0 0 0) 0x7f5a7c011b00 con 0x7f5a7406ace0 2023-06-09T04:36:16.953 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a7406ace0 msgr2=0x7f5a7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a7406ace0 0x7f5a7406d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0090b0 tx=0x7f5a7c008030 comp rx=0 tx=0).stop 2023-06-09T04:36:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a8806d220 msgr2=0x7f5a88155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:16.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a8806d220 0x7f5a88155f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800e840 tx=0x7f5a7800eb50 comp rx=0 tx=0).stop 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 shutdown_connections 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f5a7406ace0 0x7f5a7406d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a88151b30 0x7f5a88157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a8806d220 0x7f5a88155f10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 --2- 172.21.15.92:0/4242484642 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a88158920 0x7f5a88149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 >> 172.21.15.92:0/4242484642 conn(0x7f5a880b1800 msgr2=0x7f5a880c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 shutdown_connections 2023-06-09T04:36:16.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:16.952+0000 7f5a8e556700 1 -- 172.21.15.92:0/4242484642 wait complete. 2023-06-09T04:36:16.957 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-06-09T04:36:16.966 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-06-09T04:36:15.856209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220180,"kb_used_data":4564,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754540,"statfs":{"total":773094113280,"available":772868648960,"internally_reserved":0,"allocated":4673536,"data_stored":2598587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.433834"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962184+0000","last_change":"2023-06-09T04:36:12.842497+0000","last_active":"2023-06-09T04:36:13.962184+0000","last_peered":"2023-06-09T04:36:13.962184+0000","last_clean":"2023-06-09T04:36:13.962184+0000","last_became_active":"2023-06-09T04:36:12.842036+0000","last_became_peered":"2023-06-09T04:36:12.842036+0000","last_unstale":"2023-06-09T04:36:13.962184+0000","last_undegraded":"2023-06-09T04:36:13.962184+0000","last_fullsized":"2023-06-09T04:36:13.962184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:57:20.231731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961998+0000","last_change":"2023-06-09T04:36:12.838639+0000","last_active":"2023-06-09T04:36:13.961998+0000","last_peered":"2023-06-09T04:36:13.961998+0000","last_clean":"2023-06-09T04:36:13.961998+0000","last_became_active":"2023-06-09T04:36:12.838290+0000","last_became_peered":"2023-06-09T04:36:12.838290+0000","last_unstale":"2023-06-09T04:36:13.961998+0000","last_undegraded":"2023-06-09T04:36:13.961998+0000","last_fullsized":"2023-06-09T04:36:13.961998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:30:08.196210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378626+0000","last_change":"2023-06-09T04:36:12.843567+0000","last_active":"2023-06-09T04:36:14.378626+0000","last_peered":"2023-06-09T04:36:14.378626+0000","last_clean":"2023-06-09T04:36:14.378626+0000","last_became_active":"2023-06-09T04:36:12.843307+0000","last_became_peered":"2023-06-09T04:36:12.843307+0000","last_unstale":"2023-06-09T04:36:14.378626+0000","last_undegraded":"2023-06-09T04:36:14.378626+0000","last_fullsized":"2023-06-09T04:36:14.378626+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:19.355313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379412+0000","last_change":"2023-06-09T04:36:12.847798+0000","last_active":"2023-06-09T04:36:14.379412+0000","last_peered":"2023-06-09T04:36:14.379412+0000","last_clean":"2023-06-09T04:36:14.379412+0000","last_became_active":"2023-06-09T04:36:12.847547+0000","last_became_peered":"2023-06-09T04:36:12.847547+0000","last_unstale":"2023-06-09T04:36:14.379412+0000","last_undegraded":"2023-06-09T04:36:14.379412+0000","last_fullsized":"2023-06-09T04:36:14.379412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:09:40.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379141+0000","last_change":"2023-06-09T04:36:12.842862+0000","last_active":"2023-06-09T04:36:14.379141+0000","last_peered":"2023-06-09T04:36:14.379141+0000","last_clean":"2023-06-09T04:36:14.379141+0000","last_became_active":"2023-06-09T04:36:12.842617+0000","last_became_peered":"2023-06-09T04:36:12.842617+0000","last_unstale":"2023-06-09T04:36:14.379141+0000","last_undegraded":"2023-06-09T04:36:14.379141+0000","last_fullsized":"2023-06-09T04:36:14.379141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:38:56.681006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378338+0000","last_change":"2023-06-09T04:36:12.844067+0000","last_active":"2023-06-09T04:36:14.378338+0000","last_peered":"2023-06-09T04:36:14.378338+0000","last_clean":"2023-06-09T04:36:14.378338+0000","last_became_active":"2023-06-09T04:36:12.843808+0000","last_became_peered":"2023-06-09T04:36:12.843808+0000","last_unstale":"2023-06-09T04:36:14.378338+0000","last_undegraded":"2023-06-09T04:36:14.378338+0000","last_fullsized":"2023-06-09T04:36:14.378338+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:22:46.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839929+0000","last_change":"2023-06-09T04:36:12.845285+0000","last_active":"2023-06-09T04:36:13.839929+0000","last_peered":"2023-06-09T04:36:13.839929+0000","last_clean":"2023-06-09T04:36:13.839929+0000","last_became_active":"2023-06-09T04:36:12.844989+0000","last_became_peered":"2023-06-09T04:36:12.844989+0000","last_unstale":"2023-06-09T04:36:13.839929+0000","last_undegraded":"2023-06-09T04:36:13.839929+0000","last_fullsized":"2023-06-09T04:36:13.839929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:09:30.979823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961796+0000","last_change":"2023-06-09T04:36:12.841641+0000","last_active":"2023-06-09T04:36:13.961796+0000","last_peered":"2023-06-09T04:36:13.961796+0000","last_clean":"2023-06-09T04:36:13.961796+0000","last_became_active":"2023-06-09T04:36:12.841382+0000","last_became_peered":"2023-06-09T04:36:12.841382+0000","last_unstale":"2023-06-09T04:36:13.961796+0000","last_undegraded":"2023-06-09T04:36:13.961796+0000","last_fullsized":"2023-06-09T04:36:13.961796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:08:23.307929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379178+0000","last_change":"2023-06-09T04:36:12.843232+0000","last_active":"2023-06-09T04:36:14.379178+0000","last_peered":"2023-06-09T04:36:14.379178+0000","last_clean":"2023-06-09T04:36:14.379178+0000","last_became_active":"2023-06-09T04:36:12.842961+0000","last_became_peered":"2023-06-09T04:36:12.842961+0000","last_unstale":"2023-06-09T04:36:14.379178+0000","last_undegraded":"2023-06-09T04:36:14.379178+0000","last_fullsized":"2023-06-09T04:36:14.379178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:22:46.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840572+0000","last_change":"2023-06-09T04:36:12.857622+0000","last_active":"2023-06-09T04:36:13.840572+0000","last_peered":"2023-06-09T04:36:13.840572+0000","last_clean":"2023-06-09T04:36:13.840572+0000","last_became_active":"2023-06-09T04:36:12.857360+0000","last_became_peered":"2023-06-09T04:36:12.857360+0000","last_unstale":"2023-06-09T04:36:13.840572+0000","last_undegraded":"2023-06-09T04:36:13.840572+0000","last_fullsized":"2023-06-09T04:36:13.840572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:52:15.052810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840452+0000","last_change":"2023-06-09T04:36:12.846966+0000","last_active":"2023-06-09T04:36:13.840452+0000","last_peered":"2023-06-09T04:36:13.840452+0000","last_clean":"2023-06-09T04:36:13.840452+0000","last_became_active":"2023-06-09T04:36:12.846648+0000","last_became_peered":"2023-06-09T04:36:12.846648+0000","last_unstale":"2023-06-09T04:36:13.840452+0000","last_undegraded":"2023-06-09T04:36:13.840452+0000","last_fullsized":"2023-06-09T04:36:13.840452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:43.112113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379191+0000","last_change":"2023-06-09T04:36:12.837372+0000","last_active":"2023-06-09T04:36:14.379191+0000","last_peered":"2023-06-09T04:36:14.379191+0000","last_clean":"2023-06-09T04:36:14.379191+0000","last_became_active":"2023-06-09T04:36:12.837134+0000","last_became_peered":"2023-06-09T04:36:12.837134+0000","last_unstale":"2023-06-09T04:36:14.379191+0000","last_undegraded":"2023-06-09T04:36:14.379191+0000","last_fullsized":"2023-06-09T04:36:14.379191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:05:08.150735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839827+0000","last_change":"2023-06-09T04:36:12.858234+0000","last_active":"2023-06-09T04:36:13.839827+0000","last_peered":"2023-06-09T04:36:13.839827+0000","last_clean":"2023-06-09T04:36:13.839827+0000","last_became_active":"2023-06-09T04:36:12.857942+0000","last_became_peered":"2023-06-09T04:36:12.857942+0000","last_unstale":"2023-06-09T04:36:13.839827+0000","last_undegraded":"2023-06-09T04:36:13.839827+0000","last_fullsized":"2023-06-09T04:36:13.839827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:54:57.076511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378361+0000","last_change":"2023-06-09T04:36:12.852484+0000","last_active":"2023-06-09T04:36:14.378361+0000","last_peered":"2023-06-09T04:36:14.378361+0000","last_clean":"2023-06-09T04:36:14.378361+0000","last_became_active":"2023-06-09T04:36:12.851892+0000","last_became_peered":"2023-06-09T04:36:12.851892+0000","last_unstale":"2023-06-09T04:36:14.378361+0000","last_undegraded":"2023-06-09T04:36:14.378361+0000","last_fullsized":"2023-06-09T04:36:14.378361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:34:45.737696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839766+0000","last_change":"2023-06-09T04:36:12.842531+0000","last_active":"2023-06-09T04:36:13.839766+0000","last_peered":"2023-06-09T04:36:13.839766+0000","last_clean":"2023-06-09T04:36:13.839766+0000","last_became_active":"2023-06-09T04:36:12.842231+0000","last_became_peered":"2023-06-09T04:36:12.842231+0000","last_unstale":"2023-06-09T04:36:13.839766+0000","last_undegraded":"2023-06-09T04:36:13.839766+0000","last_fullsized":"2023-06-09T04:36:13.839766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:00:17.968727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839317+0000","last_change":"2023-06-09T04:36:12.857136+0000","last_active":"2023-06-09T04:36:13.839317+0000","last_peered":"2023-06-09T04:36:13.839317+0000","last_clean":"2023-06-09T04:36:13.839317+0000","last_became_active":"2023-06-09T04:36:12.856461+0000","last_became_peered":"2023-06-09T04:36:12.856461+0000","last_unstale":"2023-06-09T04:36:13.839317+0000","last_undegraded":"2023-06-09T04:36:13.839317+0000","last_fullsized":"2023-06-09T04:36:13.839317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:33:33.160369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839039+0000","last_change":"2023-06-09T04:36:12.858585+0000","last_active":"2023-06-09T04:36:13.839039+0000","last_peered":"2023-06-09T04:36:13.839039+0000","last_clean":"2023-06-09T04:36:13.839039+0000","last_became_active":"2023-06-09T04:36:12.858229+0000","last_became_peered":"2023-06-09T04:36:12.858229+0000","last_unstale":"2023-06-09T04:36:13.839039+0000","last_undegraded":"2023-06-09T04:36:13.839039+0000","last_fullsized":"2023-06-09T04:36:13.839039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:10:20.626345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837371+0000","last_change":"2023-06-09T04:36:12.849251+0000","last_active":"2023-06-09T04:36:13.837371+0000","last_peered":"2023-06-09T04:36:13.837371+0000","last_clean":"2023-06-09T04:36:13.837371+0000","last_became_active":"2023-06-09T04:36:12.848952+0000","last_became_peered":"2023-06-09T04:36:12.848952+0000","last_unstale":"2023-06-09T04:36:13.837371+0000","last_undegraded":"2023-06-09T04:36:13.837371+0000","last_fullsized":"2023-06-09T04:36:13.837371+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:59:22.971473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379524+0000","last_change":"2023-06-09T04:36:12.841527+0000","last_active":"2023-06-09T04:36:14.379524+0000","last_peered":"2023-06-09T04:36:14.379524+0000","last_clean":"2023-06-09T04:36:14.379524+0000","last_became_active":"2023-06-09T04:36:12.841251+0000","last_became_peered":"2023-06-09T04:36:12.841251+0000","last_unstale":"2023-06-09T04:36:14.379524+0000","last_undegraded":"2023-06-09T04:36:14.379524+0000","last_fullsized":"2023-06-09T04:36:14.379524+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:16:47.575343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837071+0000","last_change":"2023-06-09T04:36:12.848012+0000","last_active":"2023-06-09T04:36:13.837071+0000","last_peered":"2023-06-09T04:36:13.837071+0000","last_clean":"2023-06-09T04:36:13.837071+0000","last_became_active":"2023-06-09T04:36:12.847719+0000","last_became_peered":"2023-06-09T04:36:12.847719+0000","last_unstale":"2023-06-09T04:36:13.837071+0000","last_undegraded":"2023-06-09T04:36:13.837071+0000","last_fullsized":"2023-06-09T04:36:13.837071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:12:40.348998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839866+0000","last_change":"2023-06-09T04:36:12.853446+0000","last_active":"2023-06-09T04:36:13.839866+0000","last_peered":"2023-06-09T04:36:13.839866+0000","last_clean":"2023-06-09T04:36:13.839866+0000","last_became_active":"2023-06-09T04:36:12.852663+0000","last_became_peered":"2023-06-09T04:36:12.852663+0000","last_unstale":"2023-06-09T04:36:13.839866+0000","last_undegraded":"2023-06-09T04:36:13.839866+0000","last_fullsized":"2023-06-09T04:36:13.839866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:51:20.669127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378731+0000","last_change":"2023-06-09T04:36:12.845822+0000","last_active":"2023-06-09T04:36:14.378731+0000","last_peered":"2023-06-09T04:36:14.378731+0000","last_clean":"2023-06-09T04:36:14.378731+0000","last_became_active":"2023-06-09T04:36:12.845310+0000","last_became_peered":"2023-06-09T04:36:12.845310+0000","last_unstale":"2023-06-09T04:36:14.378731+0000","last_undegraded":"2023-06-09T04:36:14.378731+0000","last_fullsized":"2023-06-09T04:36:14.378731+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:33:33.160369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839298+0000","last_change":"2023-06-09T04:36:12.856068+0000","last_active":"2023-06-09T04:36:13.839298+0000","last_peered":"2023-06-09T04:36:13.839298+0000","last_clean":"2023-06-09T04:36:13.839298+0000","last_became_active":"2023-06-09T04:36:12.855465+0000","last_became_peered":"2023-06-09T04:36:12.855465+0000","last_unstale":"2023-06-09T04:36:13.839298+0000","last_undegraded":"2023-06-09T04:36:13.839298+0000","last_fullsized":"2023-06-09T04:36:13.839298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:03:41.465966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378461+0000","last_change":"2023-06-09T04:36:12.856039+0000","last_active":"2023-06-09T04:36:14.378461+0000","last_peered":"2023-06-09T04:36:14.378461+0000","last_clean":"2023-06-09T04:36:14.378461+0000","last_became_active":"2023-06-09T04:36:12.855800+0000","last_became_peered":"2023-06-09T04:36:12.855800+0000","last_unstale":"2023-06-09T04:36:14.378461+0000","last_undegraded":"2023-06-09T04:36:14.378461+0000","last_fullsized":"2023-06-09T04:36:14.378461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:26:00.998381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837397+0000","last_change":"2023-06-09T04:36:12.845088+0000","last_active":"2023-06-09T04:36:13.837397+0000","last_peered":"2023-06-09T04:36:13.837397+0000","last_clean":"2023-06-09T04:36:13.837397+0000","last_became_active":"2023-06-09T04:36:12.844838+0000","last_became_peered":"2023-06-09T04:36:12.844838+0000","last_unstale":"2023-06-09T04:36:13.837397+0000","last_undegraded":"2023-06-09T04:36:13.837397+0000","last_fullsized":"2023-06-09T04:36:13.837397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:38:56.681006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378788+0000","last_change":"2023-06-09T04:36:12.853078+0000","last_active":"2023-06-09T04:36:14.378788+0000","last_peered":"2023-06-09T04:36:14.378788+0000","last_clean":"2023-06-09T04:36:14.378788+0000","last_became_active":"2023-06-09T04:36:12.852835+0000","last_became_peered":"2023-06-09T04:36:12.852835+0000","last_unstale":"2023-06-09T04:36:14.378788+0000","last_undegraded":"2023-06-09T04:36:14.378788+0000","last_fullsized":"2023-06-09T04:36:14.378788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:30:08.196210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837213+0000","last_change":"2023-06-09T04:36:12.845281+0000","last_active":"2023-06-09T04:36:13.837213+0000","last_peered":"2023-06-09T04:36:13.837213+0000","last_clean":"2023-06-09T04:36:13.837213+0000","last_became_active":"2023-06-09T04:36:12.845027+0000","last_became_peered":"2023-06-09T04:36:12.845027+0000","last_unstale":"2023-06-09T04:36:13.837213+0000","last_undegraded":"2023-06-09T04:36:13.837213+0000","last_fullsized":"2023-06-09T04:36:13.837213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:26:00.998381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840236+0000","last_change":"2023-06-09T04:36:12.846037+0000","last_active":"2023-06-09T04:36:13.840236+0000","last_peered":"2023-06-09T04:36:13.840236+0000","last_clean":"2023-06-09T04:36:13.840236+0000","last_became_active":"2023-06-09T04:36:12.845751+0000","last_became_peered":"2023-06-09T04:36:12.845751+0000","last_unstale":"2023-06-09T04:36:13.840236+0000","last_undegraded":"2023-06-09T04:36:13.840236+0000","last_fullsized":"2023-06-09T04:36:13.840236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:09:09.129053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837553+0000","last_change":"2023-06-09T04:36:12.838576+0000","last_active":"2023-06-09T04:36:13.837553+0000","last_peered":"2023-06-09T04:36:13.837553+0000","last_clean":"2023-06-09T04:36:13.837553+0000","last_became_active":"2023-06-09T04:36:12.838261+0000","last_became_peered":"2023-06-09T04:36:12.838261+0000","last_unstale":"2023-06-09T04:36:13.837553+0000","last_undegraded":"2023-06-09T04:36:13.837553+0000","last_fullsized":"2023-06-09T04:36:13.837553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:40.060110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837022+0000","last_change":"2023-06-09T04:36:12.843825+0000","last_active":"2023-06-09T04:36:13.837022+0000","last_peered":"2023-06-09T04:36:13.837022+0000","last_clean":"2023-06-09T04:36:13.837022+0000","last_became_active":"2023-06-09T04:36:12.841138+0000","last_became_peered":"2023-06-09T04:36:12.841138+0000","last_unstale":"2023-06-09T04:36:13.837022+0000","last_undegraded":"2023-06-09T04:36:13.837022+0000","last_fullsized":"2023-06-09T04:36:13.837022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:18:02.143553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839181+0000","last_change":"2023-06-09T04:36:12.855726+0000","last_active":"2023-06-09T04:36:13.839181+0000","last_peered":"2023-06-09T04:36:13.839181+0000","last_clean":"2023-06-09T04:36:13.839181+0000","last_became_active":"2023-06-09T04:36:12.855132+0000","last_became_peered":"2023-06-09T04:36:12.855132+0000","last_unstale":"2023-06-09T04:36:13.839181+0000","last_undegraded":"2023-06-09T04:36:13.839181+0000","last_fullsized":"2023-06-09T04:36:13.839181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:34:45.737696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378259+0000","last_change":"2023-06-09T04:36:12.845014+0000","last_active":"2023-06-09T04:36:14.378259+0000","last_peered":"2023-06-09T04:36:14.378259+0000","last_clean":"2023-06-09T04:36:14.378259+0000","last_became_active":"2023-06-09T04:36:12.844789+0000","last_became_peered":"2023-06-09T04:36:12.844789+0000","last_unstale":"2023-06-09T04:36:14.378259+0000","last_undegraded":"2023-06-09T04:36:14.378259+0000","last_fullsized":"2023-06-09T04:36:14.378259+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:48.155051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839095+0000","last_change":"2023-06-09T04:36:12.856672+0000","last_active":"2023-06-09T04:36:13.839095+0000","last_peered":"2023-06-09T04:36:13.839095+0000","last_clean":"2023-06-09T04:36:13.839095+0000","last_became_active":"2023-06-09T04:36:12.855785+0000","last_became_peered":"2023-06-09T04:36:12.855785+0000","last_unstale":"2023-06-09T04:36:13.839095+0000","last_undegraded":"2023-06-09T04:36:13.839095+0000","last_fullsized":"2023-06-09T04:36:13.839095+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:41:54.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837549+0000","last_change":"2023-06-09T04:36:12.445926+0000","last_active":"2023-06-09T04:36:13.837549+0000","last_peered":"2023-06-09T04:36:13.837549+0000","last_clean":"2023-06-09T04:36:13.837549+0000","last_became_active":"2023-06-09T04:36:12.445684+0000","last_became_peered":"2023-06-09T04:36:12.445684+0000","last_unstale":"2023-06-09T04:36:13.837549+0000","last_undegraded":"2023-06-09T04:36:13.837549+0000","last_fullsized":"2023-06-09T04:36:13.837549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:47:46.014477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378796+0000","last_change":"2023-06-09T04:36:12.851269+0000","last_active":"2023-06-09T04:36:14.378796+0000","last_peered":"2023-06-09T04:36:14.378796+0000","last_clean":"2023-06-09T04:36:14.378796+0000","last_became_active":"2023-06-09T04:36:12.850988+0000","last_became_peered":"2023-06-09T04:36:12.850988+0000","last_unstale":"2023-06-09T04:36:14.378796+0000","last_undegraded":"2023-06-09T04:36:14.378796+0000","last_fullsized":"2023-06-09T04:36:14.378796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:53:14.997329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378879+0000","last_change":"2023-06-09T04:36:12.441147+0000","last_active":"2023-06-09T04:36:14.378879+0000","last_peered":"2023-06-09T04:36:14.378879+0000","last_clean":"2023-06-09T04:36:14.378879+0000","last_became_active":"2023-06-09T04:36:12.440866+0000","last_became_peered":"2023-06-09T04:36:12.440866+0000","last_unstale":"2023-06-09T04:36:14.378879+0000","last_undegraded":"2023-06-09T04:36:14.378879+0000","last_fullsized":"2023-06-09T04:36:14.378879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:47.152893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378762+0000","last_change":"2023-06-09T04:36:12.852148+0000","last_active":"2023-06-09T04:36:14.378762+0000","last_peered":"2023-06-09T04:36:14.378762+0000","last_clean":"2023-06-09T04:36:14.378762+0000","last_became_active":"2023-06-09T04:36:12.851336+0000","last_became_peered":"2023-06-09T04:36:12.851336+0000","last_unstale":"2023-06-09T04:36:14.378762+0000","last_undegraded":"2023-06-09T04:36:14.378762+0000","last_fullsized":"2023-06-09T04:36:14.378762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:36:05.318276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378952+0000","last_change":"2023-06-09T04:36:12.446203+0000","last_active":"2023-06-09T04:36:14.378952+0000","last_peered":"2023-06-09T04:36:14.378952+0000","last_clean":"2023-06-09T04:36:14.378952+0000","last_became_active":"2023-06-09T04:36:12.445884+0000","last_became_peered":"2023-06-09T04:36:12.445884+0000","last_unstale":"2023-06-09T04:36:14.378952+0000","last_undegraded":"2023-06-09T04:36:14.378952+0000","last_fullsized":"2023-06-09T04:36:14.378952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:02.355163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962227+0000","last_change":"2023-06-09T04:36:12.851559+0000","last_active":"2023-06-09T04:36:13.962227+0000","last_peered":"2023-06-09T04:36:13.962227+0000","last_clean":"2023-06-09T04:36:13.962227+0000","last_became_active":"2023-06-09T04:36:12.851250+0000","last_became_peered":"2023-06-09T04:36:12.851250+0000","last_unstale":"2023-06-09T04:36:13.962227+0000","last_undegraded":"2023-06-09T04:36:13.962227+0000","last_fullsized":"2023-06-09T04:36:13.962227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:30:50.035335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378565+0000","last_change":"2023-06-09T04:36:12.451987+0000","last_active":"2023-06-09T04:36:14.378565+0000","last_peered":"2023-06-09T04:36:14.378565+0000","last_clean":"2023-06-09T04:36:14.378565+0000","last_became_active":"2023-06-09T04:36:12.451770+0000","last_became_peered":"2023-06-09T04:36:12.451770+0000","last_unstale":"2023-06-09T04:36:14.378565+0000","last_undegraded":"2023-06-09T04:36:14.378565+0000","last_fullsized":"2023-06-09T04:36:14.378565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:42.109954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378485+0000","last_change":"2023-06-09T04:36:12.838731+0000","last_active":"2023-06-09T04:36:14.378485+0000","last_peered":"2023-06-09T04:36:14.378485+0000","last_clean":"2023-06-09T04:36:14.378485+0000","last_became_active":"2023-06-09T04:36:12.838493+0000","last_became_peered":"2023-06-09T04:36:12.838493+0000","last_unstale":"2023-06-09T04:36:14.378485+0000","last_undegraded":"2023-06-09T04:36:14.378485+0000","last_fullsized":"2023-06-09T04:36:14.378485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:35.972279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840012+0000","last_change":"2023-06-09T04:36:12.446570+0000","last_active":"2023-06-09T04:36:13.840012+0000","last_peered":"2023-06-09T04:36:13.840012+0000","last_clean":"2023-06-09T04:36:13.840012+0000","last_became_active":"2023-06-09T04:36:12.446257+0000","last_became_peered":"2023-06-09T04:36:12.446257+0000","last_unstale":"2023-06-09T04:36:13.840012+0000","last_undegraded":"2023-06-09T04:36:13.840012+0000","last_fullsized":"2023-06-09T04:36:13.840012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:10:19.624187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840000+0000","last_change":"2023-06-09T04:36:12.851910+0000","last_active":"2023-06-09T04:36:13.840000+0000","last_peered":"2023-06-09T04:36:13.840000+0000","last_clean":"2023-06-09T04:36:13.840000+0000","last_became_active":"2023-06-09T04:36:12.851576+0000","last_became_peered":"2023-06-09T04:36:12.851576+0000","last_unstale":"2023-06-09T04:36:13.840000+0000","last_undegraded":"2023-06-09T04:36:13.840000+0000","last_fullsized":"2023-06-09T04:36:13.840000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:54:27.601994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838967+0000","last_change":"2023-06-09T04:36:12.447988+0000","last_active":"2023-06-09T04:36:13.838967+0000","last_peered":"2023-06-09T04:36:13.838967+0000","last_clean":"2023-06-09T04:36:13.838967+0000","last_became_active":"2023-06-09T04:36:12.447717+0000","last_became_peered":"2023-06-09T04:36:12.447717+0000","last_unstale":"2023-06-09T04:36:13.838967+0000","last_undegraded":"2023-06-09T04:36:13.838967+0000","last_fullsized":"2023-06-09T04:36:13.838967+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:51:19.666968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378573+0000","last_change":"2023-06-09T04:36:12.845620+0000","last_active":"2023-06-09T04:36:14.378573+0000","last_peered":"2023-06-09T04:36:14.378573+0000","last_clean":"2023-06-09T04:36:14.378573+0000","last_became_active":"2023-06-09T04:36:12.844933+0000","last_became_peered":"2023-06-09T04:36:12.844933+0000","last_unstale":"2023-06-09T04:36:14.378573+0000","last_undegraded":"2023-06-09T04:36:14.378573+0000","last_fullsized":"2023-06-09T04:36:14.378573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:03:41.465966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378699+0000","last_change":"2023-06-09T04:36:12.444330+0000","last_active":"2023-06-09T04:36:14.378699+0000","last_peered":"2023-06-09T04:36:14.378699+0000","last_clean":"2023-06-09T04:36:14.378699+0000","last_became_active":"2023-06-09T04:36:12.439457+0000","last_became_peered":"2023-06-09T04:36:12.439457+0000","last_unstale":"2023-06-09T04:36:14.378699+0000","last_undegraded":"2023-06-09T04:36:14.378699+0000","last_fullsized":"2023-06-09T04:36:14.378699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:20:01.685732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962095+0000","last_change":"2023-06-09T04:36:12.847774+0000","last_active":"2023-06-09T04:36:13.962095+0000","last_peered":"2023-06-09T04:36:13.962095+0000","last_clean":"2023-06-09T04:36:13.962095+0000","last_became_active":"2023-06-09T04:36:12.847514+0000","last_became_peered":"2023-06-09T04:36:12.847514+0000","last_unstale":"2023-06-09T04:36:13.962095+0000","last_undegraded":"2023-06-09T04:36:13.962095+0000","last_fullsized":"2023-06-09T04:36:13.962095+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:03.357322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378326+0000","last_change":"2023-06-09T04:36:12.440208+0000","last_active":"2023-06-09T04:36:14.378326+0000","last_peered":"2023-06-09T04:36:14.378326+0000","last_clean":"2023-06-09T04:36:14.378326+0000","last_became_active":"2023-06-09T04:36:12.439881+0000","last_became_peered":"2023-06-09T04:36:12.439881+0000","last_unstale":"2023-06-09T04:36:14.378326+0000","last_undegraded":"2023-06-09T04:36:14.378326+0000","last_fullsized":"2023-06-09T04:36:14.378326+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:28:27.432581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379457+0000","last_change":"2023-06-09T04:36:12.847463+0000","last_active":"2023-06-09T04:36:14.379457+0000","last_peered":"2023-06-09T04:36:14.379457+0000","last_clean":"2023-06-09T04:36:14.379457+0000","last_became_active":"2023-06-09T04:36:12.847172+0000","last_became_peered":"2023-06-09T04:36:12.847172+0000","last_unstale":"2023-06-09T04:36:14.379457+0000","last_undegraded":"2023-06-09T04:36:14.379457+0000","last_fullsized":"2023-06-09T04:36:14.379457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:08:04.878446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962272+0000","last_change":"2023-06-09T04:36:12.441517+0000","last_active":"2023-06-09T04:36:13.962272+0000","last_peered":"2023-06-09T04:36:13.962272+0000","last_clean":"2023-06-09T04:36:13.962272+0000","last_became_active":"2023-06-09T04:36:12.441245+0000","last_became_peered":"2023-06-09T04:36:12.441245+0000","last_unstale":"2023-06-09T04:36:13.962272+0000","last_undegraded":"2023-06-09T04:36:13.962272+0000","last_fullsized":"2023-06-09T04:36:13.962272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:42:49.504944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838963+0000","last_change":"2023-06-09T04:36:12.856931+0000","last_active":"2023-06-09T04:36:13.838963+0000","last_peered":"2023-06-09T04:36:13.838963+0000","last_clean":"2023-06-09T04:36:13.838963+0000","last_became_active":"2023-06-09T04:36:12.856451+0000","last_became_peered":"2023-06-09T04:36:12.856451+0000","last_unstale":"2023-06-09T04:36:13.838963+0000","last_undegraded":"2023-06-09T04:36:13.838963+0000","last_fullsized":"2023-06-09T04:36:13.838963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:54:57.076511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840634+0000","last_change":"2023-06-09T04:36:12.441640+0000","last_active":"2023-06-09T04:36:13.840634+0000","last_peered":"2023-06-09T04:36:13.840634+0000","last_clean":"2023-06-09T04:36:13.840634+0000","last_became_active":"2023-06-09T04:36:12.441357+0000","last_became_peered":"2023-06-09T04:36:12.441357+0000","last_unstale":"2023-06-09T04:36:13.840634+0000","last_undegraded":"2023-06-09T04:36:13.840634+0000","last_fullsized":"2023-06-09T04:36:13.840634+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:27.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378781+0000","last_change":"2023-06-09T04:36:12.854423+0000","last_active":"2023-06-09T04:36:14.378781+0000","last_peered":"2023-06-09T04:36:14.378781+0000","last_clean":"2023-06-09T04:36:14.378781+0000","last_became_active":"2023-06-09T04:36:12.854073+0000","last_became_peered":"2023-06-09T04:36:12.854073+0000","last_unstale":"2023-06-09T04:36:14.378781+0000","last_undegraded":"2023-06-09T04:36:14.378781+0000","last_fullsized":"2023-06-09T04:36:14.378781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:50:53.949504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840577+0000","last_change":"2023-06-09T04:36:12.446010+0000","last_active":"2023-06-09T04:36:13.840577+0000","last_peered":"2023-06-09T04:36:13.840577+0000","last_clean":"2023-06-09T04:36:13.840577+0000","last_became_active":"2023-06-09T04:36:12.445754+0000","last_became_peered":"2023-06-09T04:36:12.445754+0000","last_unstale":"2023-06-09T04:36:13.840577+0000","last_undegraded":"2023-06-09T04:36:13.840577+0000","last_fullsized":"2023-06-09T04:36:13.840577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:01.520688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378827+0000","last_change":"2023-06-09T04:36:12.845488+0000","last_active":"2023-06-09T04:36:14.378827+0000","last_peered":"2023-06-09T04:36:14.378827+0000","last_clean":"2023-06-09T04:36:14.378827+0000","last_became_active":"2023-06-09T04:36:12.845198+0000","last_became_peered":"2023-06-09T04:36:12.845198+0000","last_unstale":"2023-06-09T04:36:14.378827+0000","last_undegraded":"2023-06-09T04:36:14.378827+0000","last_fullsized":"2023-06-09T04:36:14.378827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:28.934134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962195+0000","last_change":"2023-06-09T04:36:12.440340+0000","last_active":"2023-06-09T04:36:13.962195+0000","last_peered":"2023-06-09T04:36:13.962195+0000","last_clean":"2023-06-09T04:36:13.962195+0000","last_became_active":"2023-06-09T04:36:12.440048+0000","last_became_peered":"2023-06-09T04:36:12.440048+0000","last_unstale":"2023-06-09T04:36:13.962195+0000","last_undegraded":"2023-06-09T04:36:13.962195+0000","last_fullsized":"2023-06-09T04:36:13.962195+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:42:42.219991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378806+0000","last_change":"2023-06-09T04:36:12.843376+0000","last_active":"2023-06-09T04:36:14.378806+0000","last_peered":"2023-06-09T04:36:14.378806+0000","last_clean":"2023-06-09T04:36:14.378806+0000","last_became_active":"2023-06-09T04:36:12.843097+0000","last_became_peered":"2023-06-09T04:36:12.843097+0000","last_unstale":"2023-06-09T04:36:14.378806+0000","last_undegraded":"2023-06-09T04:36:14.378806+0000","last_fullsized":"2023-06-09T04:36:14.378806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:29:03.620214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962145+0000","last_change":"2023-06-09T04:36:12.451004+0000","last_active":"2023-06-09T04:36:13.962145+0000","last_peered":"2023-06-09T04:36:13.962145+0000","last_clean":"2023-06-09T04:36:13.962145+0000","last_became_active":"2023-06-09T04:36:12.450718+0000","last_became_peered":"2023-06-09T04:36:12.450718+0000","last_unstale":"2023-06-09T04:36:13.962145+0000","last_undegraded":"2023-06-09T04:36:13.962145+0000","last_fullsized":"2023-06-09T04:36:13.962145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:59:25.596099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961982+0000","last_change":"2023-06-09T04:36:12.840950+0000","last_active":"2023-06-09T04:36:13.961982+0000","last_peered":"2023-06-09T04:36:13.961982+0000","last_clean":"2023-06-09T04:36:13.961982+0000","last_became_active":"2023-06-09T04:36:12.840622+0000","last_became_peered":"2023-06-09T04:36:12.840622+0000","last_unstale":"2023-06-09T04:36:13.961982+0000","last_undegraded":"2023-06-09T04:36:13.961982+0000","last_fullsized":"2023-06-09T04:36:13.961982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:20:02.687890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378463+0000","last_change":"2023-06-09T04:36:12.444910+0000","last_active":"2023-06-09T04:36:14.378463+0000","last_peered":"2023-06-09T04:36:14.378463+0000","last_clean":"2023-06-09T04:36:14.378463+0000","last_became_active":"2023-06-09T04:36:12.444645+0000","last_became_peered":"2023-06-09T04:36:12.444645+0000","last_unstale":"2023-06-09T04:36:14.378463+0000","last_undegraded":"2023-06-09T04:36:14.378463+0000","last_fullsized":"2023-06-09T04:36:14.378463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:05:07.148576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839011+0000","last_change":"2023-06-09T04:36:12.855925+0000","last_active":"2023-06-09T04:36:13.839011+0000","last_peered":"2023-06-09T04:36:13.839011+0000","last_clean":"2023-06-09T04:36:13.839011+0000","last_became_active":"2023-06-09T04:36:12.847191+0000","last_became_peered":"2023-06-09T04:36:12.847191+0000","last_unstale":"2023-06-09T04:36:13.839011+0000","last_undegraded":"2023-06-09T04:36:13.839011+0000","last_fullsized":"2023-06-09T04:36:13.839011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:09:30.979823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.836964+0000","last_change":"2023-06-09T04:36:12.445941+0000","last_active":"2023-06-09T04:36:13.836964+0000","last_peered":"2023-06-09T04:36:13.836964+0000","last_clean":"2023-06-09T04:36:13.836964+0000","last_became_active":"2023-06-09T04:36:12.445651+0000","last_became_peered":"2023-06-09T04:36:12.445651+0000","last_unstale":"2023-06-09T04:36:13.836964+0000","last_undegraded":"2023-06-09T04:36:13.836964+0000","last_fullsized":"2023-06-09T04:36:13.836964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:41:56.097420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961933+0000","last_change":"2023-06-09T04:36:12.843164+0000","last_active":"2023-06-09T04:36:13.961933+0000","last_peered":"2023-06-09T04:36:13.961933+0000","last_clean":"2023-06-09T04:36:13.961933+0000","last_became_active":"2023-06-09T04:36:12.842079+0000","last_became_peered":"2023-06-09T04:36:12.842079+0000","last_unstale":"2023-06-09T04:36:13.961933+0000","last_undegraded":"2023-06-09T04:36:13.961933+0000","last_fullsized":"2023-06-09T04:36:13.961933+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:48:33.479326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839046+0000","last_change":"2023-06-09T04:36:12.448496+0000","last_active":"2023-06-09T04:36:13.839046+0000","last_peered":"2023-06-09T04:36:13.839046+0000","last_clean":"2023-06-09T04:36:13.839046+0000","last_became_active":"2023-06-09T04:36:12.448254+0000","last_became_peered":"2023-06-09T04:36:12.448254+0000","last_unstale":"2023-06-09T04:36:13.839046+0000","last_undegraded":"2023-06-09T04:36:13.839046+0000","last_fullsized":"2023-06-09T04:36:13.839046+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:54:26.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962062+0000","last_change":"2023-06-09T04:36:12.848874+0000","last_active":"2023-06-09T04:36:13.962062+0000","last_peered":"2023-06-09T04:36:13.962062+0000","last_clean":"2023-06-09T04:36:13.962062+0000","last_became_active":"2023-06-09T04:36:12.848591+0000","last_became_peered":"2023-06-09T04:36:12.848591+0000","last_unstale":"2023-06-09T04:36:13.962062+0000","last_undegraded":"2023-06-09T04:36:13.962062+0000","last_fullsized":"2023-06-09T04:36:13.962062+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:10:58.909090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840575+0000","last_change":"2023-06-09T04:36:12.448102+0000","last_active":"2023-06-09T04:36:13.840575+0000","last_peered":"2023-06-09T04:36:13.840575+0000","last_clean":"2023-06-09T04:36:13.840575+0000","last_became_active":"2023-06-09T04:36:12.447794+0000","last_became_peered":"2023-06-09T04:36:12.447794+0000","last_unstale":"2023-06-09T04:36:13.840575+0000","last_undegraded":"2023-06-09T04:36:13.840575+0000","last_fullsized":"2023-06-09T04:36:13.840575+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:33:32.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840425+0000","last_change":"2023-06-09T04:36:12.853064+0000","last_active":"2023-06-09T04:36:13.840425+0000","last_peered":"2023-06-09T04:36:13.840425+0000","last_clean":"2023-06-09T04:36:13.840425+0000","last_became_active":"2023-06-09T04:36:12.852757+0000","last_became_peered":"2023-06-09T04:36:12.852757+0000","last_unstale":"2023-06-09T04:36:13.840425+0000","last_undegraded":"2023-06-09T04:36:13.840425+0000","last_fullsized":"2023-06-09T04:36:13.840425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:20:04.167346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378638+0000","last_change":"2023-06-09T04:36:12.447550+0000","last_active":"2023-06-09T04:36:14.378638+0000","last_peered":"2023-06-09T04:36:14.378638+0000","last_clean":"2023-06-09T04:36:14.378638+0000","last_became_active":"2023-06-09T04:36:12.447303+0000","last_became_peered":"2023-06-09T04:36:12.447303+0000","last_unstale":"2023-06-09T04:36:14.378638+0000","last_undegraded":"2023-06-09T04:36:14.378638+0000","last_fullsized":"2023-06-09T04:36:14.378638+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:52:14.050652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378745+0000","last_change":"2023-06-09T04:36:12.848115+0000","last_active":"2023-06-09T04:36:14.378745+0000","last_peered":"2023-06-09T04:36:14.378745+0000","last_clean":"2023-06-09T04:36:14.378745+0000","last_became_active":"2023-06-09T04:36:12.847679+0000","last_became_peered":"2023-06-09T04:36:12.847679+0000","last_unstale":"2023-06-09T04:36:14.378745+0000","last_undegraded":"2023-06-09T04:36:14.378745+0000","last_fullsized":"2023-06-09T04:36:14.378745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:02.522846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379468+0000","last_change":"2023-06-09T04:36:12.440911+0000","last_active":"2023-06-09T04:36:14.379468+0000","last_peered":"2023-06-09T04:36:14.379468+0000","last_clean":"2023-06-09T04:36:14.379468+0000","last_became_active":"2023-06-09T04:36:12.440648+0000","last_became_peered":"2023-06-09T04:36:12.440648+0000","last_unstale":"2023-06-09T04:36:14.379468+0000","last_undegraded":"2023-06-09T04:36:14.379468+0000","last_fullsized":"2023-06-09T04:36:14.379468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:15.950827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379636+0000","last_change":"2023-06-09T04:36:12.841287+0000","last_active":"2023-06-09T04:36:14.379636+0000","last_peered":"2023-06-09T04:36:14.379636+0000","last_clean":"2023-06-09T04:36:14.379636+0000","last_became_active":"2023-06-09T04:36:12.841013+0000","last_became_peered":"2023-06-09T04:36:12.841013+0000","last_unstale":"2023-06-09T04:36:14.379636+0000","last_undegraded":"2023-06-09T04:36:14.379636+0000","last_fullsized":"2023-06-09T04:36:14.379636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:19:37.751036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378694+0000","last_change":"2023-06-09T04:36:12.449225+0000","last_active":"2023-06-09T04:36:14.378694+0000","last_peered":"2023-06-09T04:36:14.378694+0000","last_clean":"2023-06-09T04:36:14.378694+0000","last_became_active":"2023-06-09T04:36:12.448936+0000","last_became_peered":"2023-06-09T04:36:12.448936+0000","last_unstale":"2023-06-09T04:36:14.378694+0000","last_undegraded":"2023-06-09T04:36:14.378694+0000","last_fullsized":"2023-06-09T04:36:14.378694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:48:32.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838758+0000","last_change":"2023-06-09T04:36:12.856939+0000","last_active":"2023-06-09T04:36:13.838758+0000","last_peered":"2023-06-09T04:36:13.838758+0000","last_clean":"2023-06-09T04:36:13.838758+0000","last_became_active":"2023-06-09T04:36:12.856146+0000","last_became_peered":"2023-06-09T04:36:12.856146+0000","last_unstale":"2023-06-09T04:36:13.838758+0000","last_undegraded":"2023-06-09T04:36:13.838758+0000","last_fullsized":"2023-06-09T04:36:13.838758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:00:17.968727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379245+0000","last_change":"2023-06-09T04:36:12.442518+0000","last_active":"2023-06-09T04:36:14.379245+0000","last_peered":"2023-06-09T04:36:14.379245+0000","last_clean":"2023-06-09T04:36:14.379245+0000","last_became_active":"2023-06-09T04:36:12.442212+0000","last_became_peered":"2023-06-09T04:36:12.442212+0000","last_unstale":"2023-06-09T04:36:14.379245+0000","last_undegraded":"2023-06-09T04:36:14.379245+0000","last_fullsized":"2023-06-09T04:36:14.379245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:39.057952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378473+0000","last_change":"2023-06-09T04:36:12.847386+0000","last_active":"2023-06-09T04:36:14.378473+0000","last_peered":"2023-06-09T04:36:14.378473+0000","last_clean":"2023-06-09T04:36:14.378473+0000","last_became_active":"2023-06-09T04:36:12.847080+0000","last_became_peered":"2023-06-09T04:36:12.847080+0000","last_unstale":"2023-06-09T04:36:14.378473+0000","last_undegraded":"2023-06-09T04:36:14.378473+0000","last_fullsized":"2023-06-09T04:36:14.378473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:45:25.433541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379224+0000","last_change":"2023-06-09T04:36:12.444092+0000","last_active":"2023-06-09T04:36:14.379224+0000","last_peered":"2023-06-09T04:36:14.379224+0000","last_clean":"2023-06-09T04:36:14.379224+0000","last_became_active":"2023-06-09T04:36:12.443813+0000","last_became_peered":"2023-06-09T04:36:12.443813+0000","last_unstale":"2023-06-09T04:36:14.379224+0000","last_undegraded":"2023-06-09T04:36:14.379224+0000","last_fullsized":"2023-06-09T04:36:14.379224+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:47:46.014477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840111+0000","last_change":"2023-06-09T04:36:12.849800+0000","last_active":"2023-06-09T04:36:13.840111+0000","last_peered":"2023-06-09T04:36:13.840111+0000","last_clean":"2023-06-09T04:36:13.840111+0000","last_became_active":"2023-06-09T04:36:12.849510+0000","last_became_peered":"2023-06-09T04:36:12.849510+0000","last_unstale":"2023-06-09T04:36:13.840111+0000","last_undegraded":"2023-06-09T04:36:13.840111+0000","last_fullsized":"2023-06-09T04:36:13.840111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:28:28.434740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839298+0000","last_change":"2023-06-09T04:36:12.444683+0000","last_active":"2023-06-09T04:36:13.839298+0000","last_peered":"2023-06-09T04:36:13.839298+0000","last_clean":"2023-06-09T04:36:13.839298+0000","last_became_active":"2023-06-09T04:36:12.444383+0000","last_became_peered":"2023-06-09T04:36:12.444383+0000","last_unstale":"2023-06-09T04:36:13.839298+0000","last_undegraded":"2023-06-09T04:36:13.839298+0000","last_fullsized":"2023-06-09T04:36:13.839298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:52:14.050652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962028+0000","last_change":"2023-06-09T04:36:12.856814+0000","last_active":"2023-06-09T04:36:13.962028+0000","last_peered":"2023-06-09T04:36:13.962028+0000","last_clean":"2023-06-09T04:36:13.962028+0000","last_became_active":"2023-06-09T04:36:12.856618+0000","last_became_peered":"2023-06-09T04:36:12.856618+0000","last_unstale":"2023-06-09T04:36:13.962028+0000","last_undegraded":"2023-06-09T04:36:13.962028+0000","last_fullsized":"2023-06-09T04:36:13.962028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:53:14.997329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378335+0000","last_change":"2023-06-09T04:36:12.442889+0000","last_active":"2023-06-09T04:36:14.378335+0000","last_peered":"2023-06-09T04:36:14.378335+0000","last_clean":"2023-06-09T04:36:14.378335+0000","last_became_active":"2023-06-09T04:36:12.442588+0000","last_became_peered":"2023-06-09T04:36:12.442588+0000","last_unstale":"2023-06-09T04:36:14.378335+0000","last_undegraded":"2023-06-09T04:36:14.378335+0000","last_fullsized":"2023-06-09T04:36:14.378335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:12:39.346840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378627+0000","last_change":"2023-06-09T04:36:12.837435+0000","last_active":"2023-06-09T04:36:14.378627+0000","last_peered":"2023-06-09T04:36:14.378627+0000","last_clean":"2023-06-09T04:36:14.378627+0000","last_became_active":"2023-06-09T04:36:12.837129+0000","last_became_peered":"2023-06-09T04:36:12.837129+0000","last_unstale":"2023-06-09T04:36:14.378627+0000","last_undegraded":"2023-06-09T04:36:14.378627+0000","last_fullsized":"2023-06-09T04:36:14.378627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:08:35.982569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378340+0000","last_change":"2023-06-09T04:36:12.445127+0000","last_active":"2023-06-09T04:36:14.378340+0000","last_peered":"2023-06-09T04:36:14.378340+0000","last_clean":"2023-06-09T04:36:14.378340+0000","last_became_active":"2023-06-09T04:36:12.442493+0000","last_became_peered":"2023-06-09T04:36:12.442493+0000","last_unstale":"2023-06-09T04:36:14.378340+0000","last_undegraded":"2023-06-09T04:36:14.378340+0000","last_fullsized":"2023-06-09T04:36:14.378340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:29.777239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378642+0000","last_change":"2023-06-09T04:35:12.072943+0000","last_active":"2023-06-09T04:36:14.378642+0000","last_peered":"2023-06-09T04:36:14.378642+0000","last_clean":"2023-06-09T04:36:14.378642+0000","last_became_active":"2023-06-09T04:35:12.072623+0000","last_became_peered":"2023-06-09T04:35:12.072623+0000","last_unstale":"2023-06-09T04:36:14.378642+0000","last_undegraded":"2023-06-09T04:36:14.378642+0000","last_fullsized":"2023-06-09T04:36:14.378642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:13:01.027580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962039+0000","last_change":"2023-06-09T04:36:12.851869+0000","last_active":"2023-06-09T04:36:13.962039+0000","last_peered":"2023-06-09T04:36:13.962039+0000","last_clean":"2023-06-09T04:36:13.962039+0000","last_became_active":"2023-06-09T04:36:12.851709+0000","last_became_peered":"2023-06-09T04:36:12.851709+0000","last_unstale":"2023-06-09T04:36:13.962039+0000","last_undegraded":"2023-06-09T04:36:13.962039+0000","last_fullsized":"2023-06-09T04:36:13.962039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:53:35.218311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378739+0000","last_change":"2023-06-09T04:36:12.445020+0000","last_active":"2023-06-09T04:36:14.378739+0000","last_peered":"2023-06-09T04:36:14.378739+0000","last_clean":"2023-06-09T04:36:14.378739+0000","last_became_active":"2023-06-09T04:36:12.444745+0000","last_became_peered":"2023-06-09T04:36:12.444745+0000","last_unstale":"2023-06-09T04:36:14.378739+0000","last_undegraded":"2023-06-09T04:36:14.378739+0000","last_fullsized":"2023-06-09T04:36:14.378739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:19.228681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840353+0000","last_change":"2023-06-09T04:35:12.002754+0000","last_active":"2023-06-09T04:36:13.840353+0000","last_peered":"2023-06-09T04:36:13.840353+0000","last_clean":"2023-06-09T04:36:13.840353+0000","last_became_active":"2023-06-09T04:35:12.002471+0000","last_became_peered":"2023-06-09T04:35:12.002471+0000","last_unstale":"2023-06-09T04:36:13.840353+0000","last_undegraded":"2023-06-09T04:36:13.840353+0000","last_fullsized":"2023-06-09T04:36:13.840353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:44.294575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839107+0000","last_change":"2023-06-09T04:36:12.854560+0000","last_active":"2023-06-09T04:36:13.839107+0000","last_peered":"2023-06-09T04:36:13.839107+0000","last_clean":"2023-06-09T04:36:13.839107+0000","last_became_active":"2023-06-09T04:36:12.854328+0000","last_became_peered":"2023-06-09T04:36:12.854328+0000","last_unstale":"2023-06-09T04:36:13.839107+0000","last_undegraded":"2023-06-09T04:36:13.839107+0000","last_fullsized":"2023-06-09T04:36:13.839107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:28:28.434740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379568+0000","last_change":"2023-06-09T04:36:12.440160+0000","last_active":"2023-06-09T04:36:14.379568+0000","last_peered":"2023-06-09T04:36:14.379568+0000","last_clean":"2023-06-09T04:36:14.379568+0000","last_became_active":"2023-06-09T04:36:12.439837+0000","last_became_peered":"2023-06-09T04:36:12.439837+0000","last_unstale":"2023-06-09T04:36:14.379568+0000","last_undegraded":"2023-06-09T04:36:14.379568+0000","last_fullsized":"2023-06-09T04:36:14.379568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:27:27.485483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379039+0000","last_change":"2023-06-09T04:35:12.071356+0000","last_active":"2023-06-09T04:36:14.379039+0000","last_peered":"2023-06-09T04:36:14.379039+0000","last_clean":"2023-06-09T04:36:14.379039+0000","last_became_active":"2023-06-09T04:35:12.070997+0000","last_became_peered":"2023-06-09T04:35:12.070997+0000","last_unstale":"2023-06-09T04:36:14.379039+0000","last_undegraded":"2023-06-09T04:36:14.379039+0000","last_fullsized":"2023-06-09T04:36:14.379039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:58:21.528352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840498+0000","last_change":"2023-06-09T04:36:12.845301+0000","last_active":"2023-06-09T04:36:13.840498+0000","last_peered":"2023-06-09T04:36:13.840498+0000","last_clean":"2023-06-09T04:36:13.840498+0000","last_became_active":"2023-06-09T04:36:12.845013+0000","last_became_peered":"2023-06-09T04:36:12.845013+0000","last_unstale":"2023-06-09T04:36:13.840498+0000","last_undegraded":"2023-06-09T04:36:13.840498+0000","last_fullsized":"2023-06-09T04:36:13.840498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:03:41.465966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839172+0000","last_change":"2023-06-09T04:36:12.447828+0000","last_active":"2023-06-09T04:36:13.839172+0000","last_peered":"2023-06-09T04:36:13.839172+0000","last_clean":"2023-06-09T04:36:13.839172+0000","last_became_active":"2023-06-09T04:36:12.447358+0000","last_became_peered":"2023-06-09T04:36:12.447358+0000","last_unstale":"2023-06-09T04:36:13.839172+0000","last_undegraded":"2023-06-09T04:36:13.839172+0000","last_fullsized":"2023-06-09T04:36:13.839172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:09:08.126895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839951+0000","last_change":"2023-06-09T04:35:11.999027+0000","last_active":"2023-06-09T04:36:13.839951+0000","last_peered":"2023-06-09T04:36:13.839951+0000","last_clean":"2023-06-09T04:36:13.839951+0000","last_became_active":"2023-06-09T04:35:11.998723+0000","last_became_peered":"2023-06-09T04:35:11.998723+0000","last_unstale":"2023-06-09T04:36:13.839951+0000","last_undegraded":"2023-06-09T04:36:13.839951+0000","last_fullsized":"2023-06-09T04:36:13.839951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:40:53.096806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838530+0000","last_change":"2023-06-09T04:35:06.996716+0000","last_active":"2023-06-09T04:36:13.838530+0000","last_peered":"2023-06-09T04:36:13.838530+0000","last_clean":"2023-06-09T04:36:13.838530+0000","last_became_active":"2023-06-09T04:35:06.995565+0000","last_became_peered":"2023-06-09T04:35:06.995565+0000","last_unstale":"2023-06-09T04:36:13.838530+0000","last_undegraded":"2023-06-09T04:36:13.838530+0000","last_fullsized":"2023-06-09T04:36:13.838530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_clean_scrub_stamp":"2023-06-09T04:35:05.906836+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:36:55.796586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378540+0000","last_change":"2023-06-09T04:36:12.851150+0000","last_active":"2023-06-09T04:36:14.378540+0000","last_peered":"2023-06-09T04:36:14.378540+0000","last_clean":"2023-06-09T04:36:14.378540+0000","last_became_active":"2023-06-09T04:36:12.850913+0000","last_became_peered":"2023-06-09T04:36:12.850913+0000","last_unstale":"2023-06-09T04:36:14.378540+0000","last_undegraded":"2023-06-09T04:36:14.378540+0000","last_fullsized":"2023-06-09T04:36:14.378540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:08:23.307929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840609+0000","last_change":"2023-06-09T04:36:12.441599+0000","last_active":"2023-06-09T04:36:13.840609+0000","last_peered":"2023-06-09T04:36:13.840609+0000","last_clean":"2023-06-09T04:36:13.840609+0000","last_became_active":"2023-06-09T04:36:12.441324+0000","last_became_peered":"2023-06-09T04:36:12.441324+0000","last_unstale":"2023-06-09T04:36:13.840609+0000","last_undegraded":"2023-06-09T04:36:13.840609+0000","last_fullsized":"2023-06-09T04:36:13.840609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:36:04.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838487+0000","last_change":"2023-06-09T04:35:12.002654+0000","last_active":"2023-06-09T04:36:13.838487+0000","last_peered":"2023-06-09T04:36:13.838487+0000","last_clean":"2023-06-09T04:36:13.838487+0000","last_became_active":"2023-06-09T04:35:12.002320+0000","last_became_peered":"2023-06-09T04:35:12.002320+0000","last_unstale":"2023-06-09T04:36:13.838487+0000","last_undegraded":"2023-06-09T04:36:13.838487+0000","last_fullsized":"2023-06-09T04:36:13.838487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:15:00.010109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378854+0000","last_change":"2023-06-09T04:36:12.850116+0000","last_active":"2023-06-09T04:36:14.378854+0000","last_peered":"2023-06-09T04:36:14.378854+0000","last_clean":"2023-06-09T04:36:14.378854+0000","last_became_active":"2023-06-09T04:36:12.849864+0000","last_became_peered":"2023-06-09T04:36:12.849864+0000","last_unstale":"2023-06-09T04:36:14.378854+0000","last_undegraded":"2023-06-09T04:36:14.378854+0000","last_fullsized":"2023-06-09T04:36:14.378854+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:53:35.218311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837009+0000","last_change":"2023-06-09T04:36:12.442755+0000","last_active":"2023-06-09T04:36:13.837009+0000","last_peered":"2023-06-09T04:36:13.837009+0000","last_clean":"2023-06-09T04:36:13.837009+0000","last_became_active":"2023-06-09T04:36:12.442501+0000","last_became_peered":"2023-06-09T04:36:12.442501+0000","last_unstale":"2023-06-09T04:36:13.837009+0000","last_undegraded":"2023-06-09T04:36:13.837009+0000","last_fullsized":"2023-06-09T04:36:13.837009+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:38:47.022200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839190+0000","last_change":"2023-06-09T04:35:12.071967+0000","last_active":"2023-06-09T04:36:13.839190+0000","last_peered":"2023-06-09T04:36:13.839190+0000","last_clean":"2023-06-09T04:36:13.839190+0000","last_became_active":"2023-06-09T04:35:12.071565+0000","last_became_peered":"2023-06-09T04:35:12.071565+0000","last_unstale":"2023-06-09T04:36:13.839190+0000","last_undegraded":"2023-06-09T04:36:13.839190+0000","last_fullsized":"2023-06-09T04:36:13.839190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:54:41.668992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378546+0000","last_change":"2023-06-09T04:36:12.846958+0000","last_active":"2023-06-09T04:36:14.378546+0000","last_peered":"2023-06-09T04:36:14.378546+0000","last_clean":"2023-06-09T04:36:14.378546+0000","last_became_active":"2023-06-09T04:36:12.846444+0000","last_became_peered":"2023-06-09T04:36:12.846444+0000","last_unstale":"2023-06-09T04:36:14.378546+0000","last_undegraded":"2023-06-09T04:36:14.378546+0000","last_fullsized":"2023-06-09T04:36:14.378546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:20:04.167346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378352+0000","last_change":"2023-06-09T04:36:12.446815+0000","last_active":"2023-06-09T04:36:14.378352+0000","last_peered":"2023-06-09T04:36:14.378352+0000","last_clean":"2023-06-09T04:36:14.378352+0000","last_became_active":"2023-06-09T04:36:12.445288+0000","last_became_peered":"2023-06-09T04:36:12.445288+0000","last_unstale":"2023-06-09T04:36:14.378352+0000","last_undegraded":"2023-06-09T04:36:14.378352+0000","last_fullsized":"2023-06-09T04:36:14.378352+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:09:08.126895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378867+0000","last_change":"2023-06-09T04:35:12.003438+0000","last_active":"2023-06-09T04:36:14.378867+0000","last_peered":"2023-06-09T04:36:14.378867+0000","last_clean":"2023-06-09T04:36:14.378867+0000","last_became_active":"2023-06-09T04:35:12.003155+0000","last_became_peered":"2023-06-09T04:35:12.003155+0000","last_unstale":"2023-06-09T04:36:14.378867+0000","last_undegraded":"2023-06-09T04:36:14.378867+0000","last_fullsized":"2023-06-09T04:36:14.378867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:09:57.465969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839210+0000","last_change":"2023-06-09T04:36:12.856608+0000","last_active":"2023-06-09T04:36:13.839210+0000","last_peered":"2023-06-09T04:36:13.839210+0000","last_clean":"2023-06-09T04:36:13.839210+0000","last_became_active":"2023-06-09T04:36:12.856228+0000","last_became_peered":"2023-06-09T04:36:12.856228+0000","last_unstale":"2023-06-09T04:36:13.839210+0000","last_undegraded":"2023-06-09T04:36:13.839210+0000","last_fullsized":"2023-06-09T04:36:13.839210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:20:04.167346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961852+0000","last_change":"2023-06-09T04:36:12.442988+0000","last_active":"2023-06-09T04:36:13.961852+0000","last_peered":"2023-06-09T04:36:13.961852+0000","last_clean":"2023-06-09T04:36:13.961852+0000","last_became_active":"2023-06-09T04:36:12.441199+0000","last_became_peered":"2023-06-09T04:36:12.441199+0000","last_unstale":"2023-06-09T04:36:13.961852+0000","last_undegraded":"2023-06-09T04:36:13.961852+0000","last_fullsized":"2023-06-09T04:36:13.961852+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:14:01.468543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837650+0000","last_change":"2023-06-09T04:35:12.003198+0000","last_active":"2023-06-09T04:36:13.837650+0000","last_peered":"2023-06-09T04:36:13.837650+0000","last_clean":"2023-06-09T04:36:13.837650+0000","last_became_active":"2023-06-09T04:35:12.002905+0000","last_became_peered":"2023-06-09T04:35:12.002905+0000","last_unstale":"2023-06-09T04:36:13.837650+0000","last_undegraded":"2023-06-09T04:36:13.837650+0000","last_fullsized":"2023-06-09T04:36:13.837650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:08:38.903536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":1044480,"data_stored":772053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":8.4540000000000006,"5min":8.4540000000000006,"15min":8.4540000000000006},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.432}]},{"osd":6,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.505},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":180633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.371},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.496},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":173488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.312},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32400000000000001}]},{"osd":2,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.5}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.48499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":1015808,"data_stored":763131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":5.9800000000000004,"5min":5.9800000000000004,"15min":5.9800000000000004},"last":0.24099999999999999}]},{"osd":2,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":6.0250000000000004,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.0590000000000002,"5min":5.0590000000000002,"15min":5.0590000000000002},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.673,"5min":4.673,"15min":4.673},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.0220000000000002,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.34000000000000002}]},{"osd":4,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":5.9199999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.5}]},{"osd":5,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.9539999999999997,"5min":5.9539999999999997,"15min":5.9539999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.5659999999999998,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6500000000000004,"5min":4.6500000000000004,"15min":4.6500000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":6.0030000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":0.379}]},{"osd":7,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.40400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":173468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.434,"5min":0.434,"15min":0.434},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.33200000000000002}]},{"osd":1,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.318},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.308}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":450560,"data_stored":181649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.254},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.32700000000000001}]},{"osd":1,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.316},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.34399999999999997}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.496}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":174002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.45400000000000001}]},{"osd":6,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":180163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40699999999999997}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45500000000000002}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T04:36:16.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T04:36:17.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.125+0000 7f6bf1bf9700 1 Processor -- start 2023-06-09T04:36:17.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.125+0000 7f6bf1bf9700 1 -- start start 2023-06-09T04:36:17.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bf1bf9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bec0cd8f0 0x7f6bec1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bf1bf9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec0cb540 con 0x7f6bec158920 2023-06-09T04:36:17.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bf1bf9700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec159170 con 0x7f6bec158cf0 2023-06-09T04:36:17.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bf1bf9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec06d7c0 con 0x7f6bec0cd8f0 2023-06-09T04:36:17.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bebfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6bec0cd8f0 0x7f6bec1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6beb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6bec158920 0x7f6bec15cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37998/0 (socket says 172.21.15.92:37998) 2023-06-09T04:36:17.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6beb7fe700 1 -- 172.21.15.92:0/344614218 learned_addr learned my addr 172.21.15.92:0/344614218 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:17.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bebfff700 1 -- 172.21.15.92:0/344614218 >> v1:172.21.15.123:6789/0 conn(0x7f6bec158cf0 legacy=0x7f6bec0cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bebfff700 1 -- 172.21.15.92:0/344614218 >> v1:172.21.15.92:6789/0 conn(0x7f6bec158920 legacy=0x7f6bec15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bebfff700 1 -- 172.21.15.92:0/344614218 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec159450 con 0x7f6bec0cd8f0 2023-06-09T04:36:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.126+0000 7f6bea7fc700 1 -- 172.21.15.92:0/344614218 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744284828 0 0) 0x7f6bec0cb540 con 0x7f6bec158920 2023-06-09T04:36:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bebfff700 1 --2- 172.21.15.92:0/344614218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec0cd8f0 0x7f6bec1638e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009d90 tx=0x7f6be400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd754b8dc995572b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bea7fc700 1 -- 172.21.15.92:0/344614218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400e070 con 0x7f6bec0cd8f0 2023-06-09T04:36:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bea7fc700 1 -- 172.21.15.92:0/344614218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be400bbd0 con 0x7f6bec0cd8f0 2023-06-09T04:36:17.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bea7fc700 1 -- 172.21.15.92:0/344614218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4012430 con 0x7f6bec0cd8f0 2023-06-09T04:36:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/344614218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec0cd8f0 msgr2=0x7f6bec1638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/344614218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec0cd8f0 0x7f6bec1638e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009d90 tx=0x7f6be400b330 comp rx=0 tx=0).stop 2023-06-09T04:36:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/344614218 shutdown_connections 2023-06-09T04:36:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/344614218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec0cd8f0 0x7f6bec1638e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/344614218 >> 172.21.15.92:0/344614218 conn(0x7f6bec0b1800 msgr2=0x7f6bec06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/344614218 shutdown_connections 2023-06-09T04:36:17.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.127+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/344614218 wait complete. 2023-06-09T04:36:17.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.128+0000 7f6bf1bf9700 1 Processor -- start 2023-06-09T04:36:17.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6bf1bf9700 1 -- start start 2023-06-09T04:36:17.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6bf1bf9700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bec0cd8f0 0x7f6bec14e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6bf1bf9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec158920 0x7f6bec0ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6bf1bf9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bec0b1ee0 0x7f6bec0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6bf1bf9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1573c0 con 0x7f6bec158920 2023-06-09T04:36:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6bf1bf9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1567f0 con 0x7f6bec0b1ee0 2023-06-09T04:36:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6bf1bf9700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec156a90 con 0x7f6bec0cd8f0 2023-06-09T04:36:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.129+0000 7f6beaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec158920 0x7f6bec0ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec158920 0x7f6bec0ac7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39674/0 (socket says 172.21.15.92:39674) 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 -- 172.21.15.92:0/3155242103 learned_addr learned my addr 172.21.15.92:0/3155242103 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6bebfff700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bec0b1ee0 0x7f6bec0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 -- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bec0cd8f0 msgr2=0x7f6bec14e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bec0cd8f0 0x7f6bec14e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 -- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bec0b1ee0 msgr2=0x7f6bec0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bec0b1ee0 0x7f6bec0b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 -- 172.21.15.92:0/3155242103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be4009a70 con 0x7f6bec158920 2023-06-09T04:36:17.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6bebfff700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bec0b1ee0 0x7f6bec0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6beaffd700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec158920 0x7f6bec0ac7f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c700 tx=0x7f6bdc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6bd3fff700 1 -- 172.21.15.92:0/3155242103 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0052d0 con 0x7f6bec158920 2023-06-09T04:36:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6bd3fff700 1 -- 172.21.15.92:0/3155242103 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc005430 con 0x7f6bec158920 2023-06-09T04:36:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec156d70 con 0x7f6bec158920 2023-06-09T04:36:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6bd3fff700 1 -- 172.21.15.92:0/3155242103 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc003880 con 0x7f6bec158920 2023-06-09T04:36:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.130+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec0acd30 con 0x7f6bec158920 2023-06-09T04:36:17.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.131+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec0740e0 con 0x7f6bec158920 2023-06-09T04:36:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.132+0000 7f6bd3fff700 1 -- 172.21.15.92:0/3155242103 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6bdc0039e0 con 0x7f6bec158920 2023-06-09T04:36:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.132+0000 7f6bd3fff700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6bd806ace0 0x7f6bd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.132+0000 7f6bd3fff700 1 -- 172.21.15.92:0/3155242103 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6bdc0888b0 con 0x7f6bec158920 2023-06-09T04:36:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.132+0000 7f6beb7fe700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6bd806ace0 0x7f6bd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.132+0000 7f6beb7fe700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6bd806ace0 0x7f6bd806d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6be0000c00 tx=0x7f6be0007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.136+0000 7f6bd3fff700 1 -- 172.21.15.92:0/3155242103 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6bdc053a20 con 0x7f6bec158920 2023-06-09T04:36:17.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.317+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 --> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bec159730 con 0x7f6bd806ace0 2023-06-09T04:36:17.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.325+0000 7f6bd3fff700 1 -- 172.21.15.92:0/3155242103 <== mgr.4104 v2:172.21.15.92:6832/3112104013 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311412 (secure 0 0 0) 0x7f6be0007070 con 0x7f6bd806ace0 2023-06-09T04:36:17.330 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:17.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6bd806ace0 msgr2=0x7f6bd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6bd806ace0 0x7f6bd806d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6be0000c00 tx=0x7f6be0007990 comp rx=0 tx=0).stop 2023-06-09T04:36:17.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec158920 msgr2=0x7f6bec0ac7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec158920 0x7f6bec0ac7f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc00c700 tx=0x7f6bdc00cac0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 shutdown_connections 2023-06-09T04:36:17.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f6bd806ace0 0x7f6bd806d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bec0b1ee0 0x7f6bec0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bec0cd8f0 0x7f6bec14e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 --2- 172.21.15.92:0/3155242103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bec158920 0x7f6bec0ac7f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 >> 172.21.15.92:0/3155242103 conn(0x7f6bec0b1800 msgr2=0x7f6bec06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:17.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 shutdown_connections 2023-06-09T04:36:17.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.329+0000 7f6bf1bf9700 1 -- 172.21.15.92:0/3155242103 wait complete. 2023-06-09T04:36:17.333 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-06-09T04:36:17.342 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-06-09T04:36:15.856209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220180,"kb_used_data":4564,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754540,"statfs":{"total":773094113280,"available":772868648960,"internally_reserved":0,"allocated":4673536,"data_stored":2598587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.433834"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962184+0000","last_change":"2023-06-09T04:36:12.842497+0000","last_active":"2023-06-09T04:36:13.962184+0000","last_peered":"2023-06-09T04:36:13.962184+0000","last_clean":"2023-06-09T04:36:13.962184+0000","last_became_active":"2023-06-09T04:36:12.842036+0000","last_became_peered":"2023-06-09T04:36:12.842036+0000","last_unstale":"2023-06-09T04:36:13.962184+0000","last_undegraded":"2023-06-09T04:36:13.962184+0000","last_fullsized":"2023-06-09T04:36:13.962184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:57:20.231731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961998+0000","last_change":"2023-06-09T04:36:12.838639+0000","last_active":"2023-06-09T04:36:13.961998+0000","last_peered":"2023-06-09T04:36:13.961998+0000","last_clean":"2023-06-09T04:36:13.961998+0000","last_became_active":"2023-06-09T04:36:12.838290+0000","last_became_peered":"2023-06-09T04:36:12.838290+0000","last_unstale":"2023-06-09T04:36:13.961998+0000","last_undegraded":"2023-06-09T04:36:13.961998+0000","last_fullsized":"2023-06-09T04:36:13.961998+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:30:08.196210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378626+0000","last_change":"2023-06-09T04:36:12.843567+0000","last_active":"2023-06-09T04:36:14.378626+0000","last_peered":"2023-06-09T04:36:14.378626+0000","last_clean":"2023-06-09T04:36:14.378626+0000","last_became_active":"2023-06-09T04:36:12.843307+0000","last_became_peered":"2023-06-09T04:36:12.843307+0000","last_unstale":"2023-06-09T04:36:14.378626+0000","last_undegraded":"2023-06-09T04:36:14.378626+0000","last_fullsized":"2023-06-09T04:36:14.378626+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:39:19.355313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379412+0000","last_change":"2023-06-09T04:36:12.847798+0000","last_active":"2023-06-09T04:36:14.379412+0000","last_peered":"2023-06-09T04:36:14.379412+0000","last_clean":"2023-06-09T04:36:14.379412+0000","last_became_active":"2023-06-09T04:36:12.847547+0000","last_became_peered":"2023-06-09T04:36:12.847547+0000","last_unstale":"2023-06-09T04:36:14.379412+0000","last_undegraded":"2023-06-09T04:36:14.379412+0000","last_fullsized":"2023-06-09T04:36:14.379412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:09:40.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379141+0000","last_change":"2023-06-09T04:36:12.842862+0000","last_active":"2023-06-09T04:36:14.379141+0000","last_peered":"2023-06-09T04:36:14.379141+0000","last_clean":"2023-06-09T04:36:14.379141+0000","last_became_active":"2023-06-09T04:36:12.842617+0000","last_became_peered":"2023-06-09T04:36:12.842617+0000","last_unstale":"2023-06-09T04:36:14.379141+0000","last_undegraded":"2023-06-09T04:36:14.379141+0000","last_fullsized":"2023-06-09T04:36:14.379141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:38:56.681006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378338+0000","last_change":"2023-06-09T04:36:12.844067+0000","last_active":"2023-06-09T04:36:14.378338+0000","last_peered":"2023-06-09T04:36:14.378338+0000","last_clean":"2023-06-09T04:36:14.378338+0000","last_became_active":"2023-06-09T04:36:12.843808+0000","last_became_peered":"2023-06-09T04:36:12.843808+0000","last_unstale":"2023-06-09T04:36:14.378338+0000","last_undegraded":"2023-06-09T04:36:14.378338+0000","last_fullsized":"2023-06-09T04:36:14.378338+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:22:46.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839929+0000","last_change":"2023-06-09T04:36:12.845285+0000","last_active":"2023-06-09T04:36:13.839929+0000","last_peered":"2023-06-09T04:36:13.839929+0000","last_clean":"2023-06-09T04:36:13.839929+0000","last_became_active":"2023-06-09T04:36:12.844989+0000","last_became_peered":"2023-06-09T04:36:12.844989+0000","last_unstale":"2023-06-09T04:36:13.839929+0000","last_undegraded":"2023-06-09T04:36:13.839929+0000","last_fullsized":"2023-06-09T04:36:13.839929+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:09:30.979823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961796+0000","last_change":"2023-06-09T04:36:12.841641+0000","last_active":"2023-06-09T04:36:13.961796+0000","last_peered":"2023-06-09T04:36:13.961796+0000","last_clean":"2023-06-09T04:36:13.961796+0000","last_became_active":"2023-06-09T04:36:12.841382+0000","last_became_peered":"2023-06-09T04:36:12.841382+0000","last_unstale":"2023-06-09T04:36:13.961796+0000","last_undegraded":"2023-06-09T04:36:13.961796+0000","last_fullsized":"2023-06-09T04:36:13.961796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:08:23.307929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379178+0000","last_change":"2023-06-09T04:36:12.843232+0000","last_active":"2023-06-09T04:36:14.379178+0000","last_peered":"2023-06-09T04:36:14.379178+0000","last_clean":"2023-06-09T04:36:14.379178+0000","last_became_active":"2023-06-09T04:36:12.842961+0000","last_became_peered":"2023-06-09T04:36:12.842961+0000","last_unstale":"2023-06-09T04:36:14.379178+0000","last_undegraded":"2023-06-09T04:36:14.379178+0000","last_fullsized":"2023-06-09T04:36:14.379178+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:22:46.405525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840572+0000","last_change":"2023-06-09T04:36:12.857622+0000","last_active":"2023-06-09T04:36:13.840572+0000","last_peered":"2023-06-09T04:36:13.840572+0000","last_clean":"2023-06-09T04:36:13.840572+0000","last_became_active":"2023-06-09T04:36:12.857360+0000","last_became_peered":"2023-06-09T04:36:12.857360+0000","last_unstale":"2023-06-09T04:36:13.840572+0000","last_undegraded":"2023-06-09T04:36:13.840572+0000","last_fullsized":"2023-06-09T04:36:13.840572+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:52:15.052810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840452+0000","last_change":"2023-06-09T04:36:12.846966+0000","last_active":"2023-06-09T04:36:13.840452+0000","last_peered":"2023-06-09T04:36:13.840452+0000","last_clean":"2023-06-09T04:36:13.840452+0000","last_became_active":"2023-06-09T04:36:12.846648+0000","last_became_peered":"2023-06-09T04:36:12.846648+0000","last_unstale":"2023-06-09T04:36:13.840452+0000","last_undegraded":"2023-06-09T04:36:13.840452+0000","last_fullsized":"2023-06-09T04:36:13.840452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:43.112113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379191+0000","last_change":"2023-06-09T04:36:12.837372+0000","last_active":"2023-06-09T04:36:14.379191+0000","last_peered":"2023-06-09T04:36:14.379191+0000","last_clean":"2023-06-09T04:36:14.379191+0000","last_became_active":"2023-06-09T04:36:12.837134+0000","last_became_peered":"2023-06-09T04:36:12.837134+0000","last_unstale":"2023-06-09T04:36:14.379191+0000","last_undegraded":"2023-06-09T04:36:14.379191+0000","last_fullsized":"2023-06-09T04:36:14.379191+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:05:08.150735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839827+0000","last_change":"2023-06-09T04:36:12.858234+0000","last_active":"2023-06-09T04:36:13.839827+0000","last_peered":"2023-06-09T04:36:13.839827+0000","last_clean":"2023-06-09T04:36:13.839827+0000","last_became_active":"2023-06-09T04:36:12.857942+0000","last_became_peered":"2023-06-09T04:36:12.857942+0000","last_unstale":"2023-06-09T04:36:13.839827+0000","last_undegraded":"2023-06-09T04:36:13.839827+0000","last_fullsized":"2023-06-09T04:36:13.839827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:54:57.076511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378361+0000","last_change":"2023-06-09T04:36:12.852484+0000","last_active":"2023-06-09T04:36:14.378361+0000","last_peered":"2023-06-09T04:36:14.378361+0000","last_clean":"2023-06-09T04:36:14.378361+0000","last_became_active":"2023-06-09T04:36:12.851892+0000","last_became_peered":"2023-06-09T04:36:12.851892+0000","last_unstale":"2023-06-09T04:36:14.378361+0000","last_undegraded":"2023-06-09T04:36:14.378361+0000","last_fullsized":"2023-06-09T04:36:14.378361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:34:45.737696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839766+0000","last_change":"2023-06-09T04:36:12.842531+0000","last_active":"2023-06-09T04:36:13.839766+0000","last_peered":"2023-06-09T04:36:13.839766+0000","last_clean":"2023-06-09T04:36:13.839766+0000","last_became_active":"2023-06-09T04:36:12.842231+0000","last_became_peered":"2023-06-09T04:36:12.842231+0000","last_unstale":"2023-06-09T04:36:13.839766+0000","last_undegraded":"2023-06-09T04:36:13.839766+0000","last_fullsized":"2023-06-09T04:36:13.839766+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:00:17.968727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839317+0000","last_change":"2023-06-09T04:36:12.857136+0000","last_active":"2023-06-09T04:36:13.839317+0000","last_peered":"2023-06-09T04:36:13.839317+0000","last_clean":"2023-06-09T04:36:13.839317+0000","last_became_active":"2023-06-09T04:36:12.856461+0000","last_became_peered":"2023-06-09T04:36:12.856461+0000","last_unstale":"2023-06-09T04:36:13.839317+0000","last_undegraded":"2023-06-09T04:36:13.839317+0000","last_fullsized":"2023-06-09T04:36:13.839317+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:33:33.160369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839039+0000","last_change":"2023-06-09T04:36:12.858585+0000","last_active":"2023-06-09T04:36:13.839039+0000","last_peered":"2023-06-09T04:36:13.839039+0000","last_clean":"2023-06-09T04:36:13.839039+0000","last_became_active":"2023-06-09T04:36:12.858229+0000","last_became_peered":"2023-06-09T04:36:12.858229+0000","last_unstale":"2023-06-09T04:36:13.839039+0000","last_undegraded":"2023-06-09T04:36:13.839039+0000","last_fullsized":"2023-06-09T04:36:13.839039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:10:20.626345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837371+0000","last_change":"2023-06-09T04:36:12.849251+0000","last_active":"2023-06-09T04:36:13.837371+0000","last_peered":"2023-06-09T04:36:13.837371+0000","last_clean":"2023-06-09T04:36:13.837371+0000","last_became_active":"2023-06-09T04:36:12.848952+0000","last_became_peered":"2023-06-09T04:36:12.848952+0000","last_unstale":"2023-06-09T04:36:13.837371+0000","last_undegraded":"2023-06-09T04:36:13.837371+0000","last_fullsized":"2023-06-09T04:36:13.837371+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:59:22.971473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379524+0000","last_change":"2023-06-09T04:36:12.841527+0000","last_active":"2023-06-09T04:36:14.379524+0000","last_peered":"2023-06-09T04:36:14.379524+0000","last_clean":"2023-06-09T04:36:14.379524+0000","last_became_active":"2023-06-09T04:36:12.841251+0000","last_became_peered":"2023-06-09T04:36:12.841251+0000","last_unstale":"2023-06-09T04:36:14.379524+0000","last_undegraded":"2023-06-09T04:36:14.379524+0000","last_fullsized":"2023-06-09T04:36:14.379524+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:16:47.575343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837071+0000","last_change":"2023-06-09T04:36:12.848012+0000","last_active":"2023-06-09T04:36:13.837071+0000","last_peered":"2023-06-09T04:36:13.837071+0000","last_clean":"2023-06-09T04:36:13.837071+0000","last_became_active":"2023-06-09T04:36:12.847719+0000","last_became_peered":"2023-06-09T04:36:12.847719+0000","last_unstale":"2023-06-09T04:36:13.837071+0000","last_undegraded":"2023-06-09T04:36:13.837071+0000","last_fullsized":"2023-06-09T04:36:13.837071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:12:40.348998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839866+0000","last_change":"2023-06-09T04:36:12.853446+0000","last_active":"2023-06-09T04:36:13.839866+0000","last_peered":"2023-06-09T04:36:13.839866+0000","last_clean":"2023-06-09T04:36:13.839866+0000","last_became_active":"2023-06-09T04:36:12.852663+0000","last_became_peered":"2023-06-09T04:36:12.852663+0000","last_unstale":"2023-06-09T04:36:13.839866+0000","last_undegraded":"2023-06-09T04:36:13.839866+0000","last_fullsized":"2023-06-09T04:36:13.839866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:51:20.669127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378731+0000","last_change":"2023-06-09T04:36:12.845822+0000","last_active":"2023-06-09T04:36:14.378731+0000","last_peered":"2023-06-09T04:36:14.378731+0000","last_clean":"2023-06-09T04:36:14.378731+0000","last_became_active":"2023-06-09T04:36:12.845310+0000","last_became_peered":"2023-06-09T04:36:12.845310+0000","last_unstale":"2023-06-09T04:36:14.378731+0000","last_undegraded":"2023-06-09T04:36:14.378731+0000","last_fullsized":"2023-06-09T04:36:14.378731+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:33:33.160369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839298+0000","last_change":"2023-06-09T04:36:12.856068+0000","last_active":"2023-06-09T04:36:13.839298+0000","last_peered":"2023-06-09T04:36:13.839298+0000","last_clean":"2023-06-09T04:36:13.839298+0000","last_became_active":"2023-06-09T04:36:12.855465+0000","last_became_peered":"2023-06-09T04:36:12.855465+0000","last_unstale":"2023-06-09T04:36:13.839298+0000","last_undegraded":"2023-06-09T04:36:13.839298+0000","last_fullsized":"2023-06-09T04:36:13.839298+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:03:41.465966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378461+0000","last_change":"2023-06-09T04:36:12.856039+0000","last_active":"2023-06-09T04:36:14.378461+0000","last_peered":"2023-06-09T04:36:14.378461+0000","last_clean":"2023-06-09T04:36:14.378461+0000","last_became_active":"2023-06-09T04:36:12.855800+0000","last_became_peered":"2023-06-09T04:36:12.855800+0000","last_unstale":"2023-06-09T04:36:14.378461+0000","last_undegraded":"2023-06-09T04:36:14.378461+0000","last_fullsized":"2023-06-09T04:36:14.378461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:26:00.998381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837397+0000","last_change":"2023-06-09T04:36:12.845088+0000","last_active":"2023-06-09T04:36:13.837397+0000","last_peered":"2023-06-09T04:36:13.837397+0000","last_clean":"2023-06-09T04:36:13.837397+0000","last_became_active":"2023-06-09T04:36:12.844838+0000","last_became_peered":"2023-06-09T04:36:12.844838+0000","last_unstale":"2023-06-09T04:36:13.837397+0000","last_undegraded":"2023-06-09T04:36:13.837397+0000","last_fullsized":"2023-06-09T04:36:13.837397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:38:56.681006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378788+0000","last_change":"2023-06-09T04:36:12.853078+0000","last_active":"2023-06-09T04:36:14.378788+0000","last_peered":"2023-06-09T04:36:14.378788+0000","last_clean":"2023-06-09T04:36:14.378788+0000","last_became_active":"2023-06-09T04:36:12.852835+0000","last_became_peered":"2023-06-09T04:36:12.852835+0000","last_unstale":"2023-06-09T04:36:14.378788+0000","last_undegraded":"2023-06-09T04:36:14.378788+0000","last_fullsized":"2023-06-09T04:36:14.378788+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:30:08.196210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837213+0000","last_change":"2023-06-09T04:36:12.845281+0000","last_active":"2023-06-09T04:36:13.837213+0000","last_peered":"2023-06-09T04:36:13.837213+0000","last_clean":"2023-06-09T04:36:13.837213+0000","last_became_active":"2023-06-09T04:36:12.845027+0000","last_became_peered":"2023-06-09T04:36:12.845027+0000","last_unstale":"2023-06-09T04:36:13.837213+0000","last_undegraded":"2023-06-09T04:36:13.837213+0000","last_fullsized":"2023-06-09T04:36:13.837213+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:26:00.998381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840236+0000","last_change":"2023-06-09T04:36:12.846037+0000","last_active":"2023-06-09T04:36:13.840236+0000","last_peered":"2023-06-09T04:36:13.840236+0000","last_clean":"2023-06-09T04:36:13.840236+0000","last_became_active":"2023-06-09T04:36:12.845751+0000","last_became_peered":"2023-06-09T04:36:12.845751+0000","last_unstale":"2023-06-09T04:36:13.840236+0000","last_undegraded":"2023-06-09T04:36:13.840236+0000","last_fullsized":"2023-06-09T04:36:13.840236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:09:09.129053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837553+0000","last_change":"2023-06-09T04:36:12.838576+0000","last_active":"2023-06-09T04:36:13.837553+0000","last_peered":"2023-06-09T04:36:13.837553+0000","last_clean":"2023-06-09T04:36:13.837553+0000","last_became_active":"2023-06-09T04:36:12.838261+0000","last_became_peered":"2023-06-09T04:36:12.838261+0000","last_unstale":"2023-06-09T04:36:13.837553+0000","last_undegraded":"2023-06-09T04:36:13.837553+0000","last_fullsized":"2023-06-09T04:36:13.837553+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:40.060110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837022+0000","last_change":"2023-06-09T04:36:12.843825+0000","last_active":"2023-06-09T04:36:13.837022+0000","last_peered":"2023-06-09T04:36:13.837022+0000","last_clean":"2023-06-09T04:36:13.837022+0000","last_became_active":"2023-06-09T04:36:12.841138+0000","last_became_peered":"2023-06-09T04:36:12.841138+0000","last_unstale":"2023-06-09T04:36:13.837022+0000","last_undegraded":"2023-06-09T04:36:13.837022+0000","last_fullsized":"2023-06-09T04:36:13.837022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:18:02.143553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839181+0000","last_change":"2023-06-09T04:36:12.855726+0000","last_active":"2023-06-09T04:36:13.839181+0000","last_peered":"2023-06-09T04:36:13.839181+0000","last_clean":"2023-06-09T04:36:13.839181+0000","last_became_active":"2023-06-09T04:36:12.855132+0000","last_became_peered":"2023-06-09T04:36:12.855132+0000","last_unstale":"2023-06-09T04:36:13.839181+0000","last_undegraded":"2023-06-09T04:36:13.839181+0000","last_fullsized":"2023-06-09T04:36:13.839181+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:34:45.737696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378259+0000","last_change":"2023-06-09T04:36:12.845014+0000","last_active":"2023-06-09T04:36:14.378259+0000","last_peered":"2023-06-09T04:36:14.378259+0000","last_clean":"2023-06-09T04:36:14.378259+0000","last_became_active":"2023-06-09T04:36:12.844789+0000","last_became_peered":"2023-06-09T04:36:12.844789+0000","last_unstale":"2023-06-09T04:36:14.378259+0000","last_undegraded":"2023-06-09T04:36:14.378259+0000","last_fullsized":"2023-06-09T04:36:14.378259+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:48.155051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839095+0000","last_change":"2023-06-09T04:36:12.856672+0000","last_active":"2023-06-09T04:36:13.839095+0000","last_peered":"2023-06-09T04:36:13.839095+0000","last_clean":"2023-06-09T04:36:13.839095+0000","last_became_active":"2023-06-09T04:36:12.855785+0000","last_became_peered":"2023-06-09T04:36:12.855785+0000","last_unstale":"2023-06-09T04:36:13.839095+0000","last_undegraded":"2023-06-09T04:36:13.839095+0000","last_fullsized":"2023-06-09T04:36:13.839095+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:41:54.539927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837549+0000","last_change":"2023-06-09T04:36:12.445926+0000","last_active":"2023-06-09T04:36:13.837549+0000","last_peered":"2023-06-09T04:36:13.837549+0000","last_clean":"2023-06-09T04:36:13.837549+0000","last_became_active":"2023-06-09T04:36:12.445684+0000","last_became_peered":"2023-06-09T04:36:12.445684+0000","last_unstale":"2023-06-09T04:36:13.837549+0000","last_undegraded":"2023-06-09T04:36:13.837549+0000","last_fullsized":"2023-06-09T04:36:13.837549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:47:46.014477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378796+0000","last_change":"2023-06-09T04:36:12.851269+0000","last_active":"2023-06-09T04:36:14.378796+0000","last_peered":"2023-06-09T04:36:14.378796+0000","last_clean":"2023-06-09T04:36:14.378796+0000","last_became_active":"2023-06-09T04:36:12.850988+0000","last_became_peered":"2023-06-09T04:36:12.850988+0000","last_unstale":"2023-06-09T04:36:14.378796+0000","last_undegraded":"2023-06-09T04:36:14.378796+0000","last_fullsized":"2023-06-09T04:36:14.378796+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:53:14.997329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378879+0000","last_change":"2023-06-09T04:36:12.441147+0000","last_active":"2023-06-09T04:36:14.378879+0000","last_peered":"2023-06-09T04:36:14.378879+0000","last_clean":"2023-06-09T04:36:14.378879+0000","last_became_active":"2023-06-09T04:36:12.440866+0000","last_became_peered":"2023-06-09T04:36:12.440866+0000","last_unstale":"2023-06-09T04:36:14.378879+0000","last_undegraded":"2023-06-09T04:36:14.378879+0000","last_fullsized":"2023-06-09T04:36:14.378879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:47.152893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378762+0000","last_change":"2023-06-09T04:36:12.852148+0000","last_active":"2023-06-09T04:36:14.378762+0000","last_peered":"2023-06-09T04:36:14.378762+0000","last_clean":"2023-06-09T04:36:14.378762+0000","last_became_active":"2023-06-09T04:36:12.851336+0000","last_became_peered":"2023-06-09T04:36:12.851336+0000","last_unstale":"2023-06-09T04:36:14.378762+0000","last_undegraded":"2023-06-09T04:36:14.378762+0000","last_fullsized":"2023-06-09T04:36:14.378762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:36:05.318276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378952+0000","last_change":"2023-06-09T04:36:12.446203+0000","last_active":"2023-06-09T04:36:14.378952+0000","last_peered":"2023-06-09T04:36:14.378952+0000","last_clean":"2023-06-09T04:36:14.378952+0000","last_became_active":"2023-06-09T04:36:12.445884+0000","last_became_peered":"2023-06-09T04:36:12.445884+0000","last_unstale":"2023-06-09T04:36:14.378952+0000","last_undegraded":"2023-06-09T04:36:14.378952+0000","last_fullsized":"2023-06-09T04:36:14.378952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:02.355163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962227+0000","last_change":"2023-06-09T04:36:12.851559+0000","last_active":"2023-06-09T04:36:13.962227+0000","last_peered":"2023-06-09T04:36:13.962227+0000","last_clean":"2023-06-09T04:36:13.962227+0000","last_became_active":"2023-06-09T04:36:12.851250+0000","last_became_peered":"2023-06-09T04:36:12.851250+0000","last_unstale":"2023-06-09T04:36:13.962227+0000","last_undegraded":"2023-06-09T04:36:13.962227+0000","last_fullsized":"2023-06-09T04:36:13.962227+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:30:50.035335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378565+0000","last_change":"2023-06-09T04:36:12.451987+0000","last_active":"2023-06-09T04:36:14.378565+0000","last_peered":"2023-06-09T04:36:14.378565+0000","last_clean":"2023-06-09T04:36:14.378565+0000","last_became_active":"2023-06-09T04:36:12.451770+0000","last_became_peered":"2023-06-09T04:36:12.451770+0000","last_unstale":"2023-06-09T04:36:14.378565+0000","last_undegraded":"2023-06-09T04:36:14.378565+0000","last_fullsized":"2023-06-09T04:36:14.378565+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:55:42.109954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378485+0000","last_change":"2023-06-09T04:36:12.838731+0000","last_active":"2023-06-09T04:36:14.378485+0000","last_peered":"2023-06-09T04:36:14.378485+0000","last_clean":"2023-06-09T04:36:14.378485+0000","last_became_active":"2023-06-09T04:36:12.838493+0000","last_became_peered":"2023-06-09T04:36:12.838493+0000","last_unstale":"2023-06-09T04:36:14.378485+0000","last_undegraded":"2023-06-09T04:36:14.378485+0000","last_fullsized":"2023-06-09T04:36:14.378485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:51:35.972279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840012+0000","last_change":"2023-06-09T04:36:12.446570+0000","last_active":"2023-06-09T04:36:13.840012+0000","last_peered":"2023-06-09T04:36:13.840012+0000","last_clean":"2023-06-09T04:36:13.840012+0000","last_became_active":"2023-06-09T04:36:12.446257+0000","last_became_peered":"2023-06-09T04:36:12.446257+0000","last_unstale":"2023-06-09T04:36:13.840012+0000","last_undegraded":"2023-06-09T04:36:13.840012+0000","last_fullsized":"2023-06-09T04:36:13.840012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:10:19.624187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840000+0000","last_change":"2023-06-09T04:36:12.851910+0000","last_active":"2023-06-09T04:36:13.840000+0000","last_peered":"2023-06-09T04:36:13.840000+0000","last_clean":"2023-06-09T04:36:13.840000+0000","last_became_active":"2023-06-09T04:36:12.851576+0000","last_became_peered":"2023-06-09T04:36:12.851576+0000","last_unstale":"2023-06-09T04:36:13.840000+0000","last_undegraded":"2023-06-09T04:36:13.840000+0000","last_fullsized":"2023-06-09T04:36:13.840000+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:54:27.601994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838967+0000","last_change":"2023-06-09T04:36:12.447988+0000","last_active":"2023-06-09T04:36:13.838967+0000","last_peered":"2023-06-09T04:36:13.838967+0000","last_clean":"2023-06-09T04:36:13.838967+0000","last_became_active":"2023-06-09T04:36:12.447717+0000","last_became_peered":"2023-06-09T04:36:12.447717+0000","last_unstale":"2023-06-09T04:36:13.838967+0000","last_undegraded":"2023-06-09T04:36:13.838967+0000","last_fullsized":"2023-06-09T04:36:13.838967+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:51:19.666968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378573+0000","last_change":"2023-06-09T04:36:12.845620+0000","last_active":"2023-06-09T04:36:14.378573+0000","last_peered":"2023-06-09T04:36:14.378573+0000","last_clean":"2023-06-09T04:36:14.378573+0000","last_became_active":"2023-06-09T04:36:12.844933+0000","last_became_peered":"2023-06-09T04:36:12.844933+0000","last_unstale":"2023-06-09T04:36:14.378573+0000","last_undegraded":"2023-06-09T04:36:14.378573+0000","last_fullsized":"2023-06-09T04:36:14.378573+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:03:41.465966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378699+0000","last_change":"2023-06-09T04:36:12.444330+0000","last_active":"2023-06-09T04:36:14.378699+0000","last_peered":"2023-06-09T04:36:14.378699+0000","last_clean":"2023-06-09T04:36:14.378699+0000","last_became_active":"2023-06-09T04:36:12.439457+0000","last_became_peered":"2023-06-09T04:36:12.439457+0000","last_unstale":"2023-06-09T04:36:14.378699+0000","last_undegraded":"2023-06-09T04:36:14.378699+0000","last_fullsized":"2023-06-09T04:36:14.378699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:20:01.685732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962095+0000","last_change":"2023-06-09T04:36:12.847774+0000","last_active":"2023-06-09T04:36:13.962095+0000","last_peered":"2023-06-09T04:36:13.962095+0000","last_clean":"2023-06-09T04:36:13.962095+0000","last_became_active":"2023-06-09T04:36:12.847514+0000","last_became_peered":"2023-06-09T04:36:12.847514+0000","last_unstale":"2023-06-09T04:36:13.962095+0000","last_undegraded":"2023-06-09T04:36:13.962095+0000","last_fullsized":"2023-06-09T04:36:13.962095+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:03.357322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378326+0000","last_change":"2023-06-09T04:36:12.440208+0000","last_active":"2023-06-09T04:36:14.378326+0000","last_peered":"2023-06-09T04:36:14.378326+0000","last_clean":"2023-06-09T04:36:14.378326+0000","last_became_active":"2023-06-09T04:36:12.439881+0000","last_became_peered":"2023-06-09T04:36:12.439881+0000","last_unstale":"2023-06-09T04:36:14.378326+0000","last_undegraded":"2023-06-09T04:36:14.378326+0000","last_fullsized":"2023-06-09T04:36:14.378326+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:28:27.432581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379457+0000","last_change":"2023-06-09T04:36:12.847463+0000","last_active":"2023-06-09T04:36:14.379457+0000","last_peered":"2023-06-09T04:36:14.379457+0000","last_clean":"2023-06-09T04:36:14.379457+0000","last_became_active":"2023-06-09T04:36:12.847172+0000","last_became_peered":"2023-06-09T04:36:12.847172+0000","last_unstale":"2023-06-09T04:36:14.379457+0000","last_undegraded":"2023-06-09T04:36:14.379457+0000","last_fullsized":"2023-06-09T04:36:14.379457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:08:04.878446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962272+0000","last_change":"2023-06-09T04:36:12.441517+0000","last_active":"2023-06-09T04:36:13.962272+0000","last_peered":"2023-06-09T04:36:13.962272+0000","last_clean":"2023-06-09T04:36:13.962272+0000","last_became_active":"2023-06-09T04:36:12.441245+0000","last_became_peered":"2023-06-09T04:36:12.441245+0000","last_unstale":"2023-06-09T04:36:13.962272+0000","last_undegraded":"2023-06-09T04:36:13.962272+0000","last_fullsized":"2023-06-09T04:36:13.962272+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:42:49.504944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838963+0000","last_change":"2023-06-09T04:36:12.856931+0000","last_active":"2023-06-09T04:36:13.838963+0000","last_peered":"2023-06-09T04:36:13.838963+0000","last_clean":"2023-06-09T04:36:13.838963+0000","last_became_active":"2023-06-09T04:36:12.856451+0000","last_became_peered":"2023-06-09T04:36:12.856451+0000","last_unstale":"2023-06-09T04:36:13.838963+0000","last_undegraded":"2023-06-09T04:36:13.838963+0000","last_fullsized":"2023-06-09T04:36:13.838963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:54:57.076511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840634+0000","last_change":"2023-06-09T04:36:12.441640+0000","last_active":"2023-06-09T04:36:13.840634+0000","last_peered":"2023-06-09T04:36:13.840634+0000","last_clean":"2023-06-09T04:36:13.840634+0000","last_became_active":"2023-06-09T04:36:12.441357+0000","last_became_peered":"2023-06-09T04:36:12.441357+0000","last_unstale":"2023-06-09T04:36:13.840634+0000","last_undegraded":"2023-06-09T04:36:13.840634+0000","last_fullsized":"2023-06-09T04:36:13.840634+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:27.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378781+0000","last_change":"2023-06-09T04:36:12.854423+0000","last_active":"2023-06-09T04:36:14.378781+0000","last_peered":"2023-06-09T04:36:14.378781+0000","last_clean":"2023-06-09T04:36:14.378781+0000","last_became_active":"2023-06-09T04:36:12.854073+0000","last_became_peered":"2023-06-09T04:36:12.854073+0000","last_unstale":"2023-06-09T04:36:14.378781+0000","last_undegraded":"2023-06-09T04:36:14.378781+0000","last_fullsized":"2023-06-09T04:36:14.378781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:50:53.949504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840577+0000","last_change":"2023-06-09T04:36:12.446010+0000","last_active":"2023-06-09T04:36:13.840577+0000","last_peered":"2023-06-09T04:36:13.840577+0000","last_clean":"2023-06-09T04:36:13.840577+0000","last_became_active":"2023-06-09T04:36:12.445754+0000","last_became_peered":"2023-06-09T04:36:12.445754+0000","last_unstale":"2023-06-09T04:36:13.840577+0000","last_undegraded":"2023-06-09T04:36:13.840577+0000","last_fullsized":"2023-06-09T04:36:13.840577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:01.520688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378827+0000","last_change":"2023-06-09T04:36:12.845488+0000","last_active":"2023-06-09T04:36:14.378827+0000","last_peered":"2023-06-09T04:36:14.378827+0000","last_clean":"2023-06-09T04:36:14.378827+0000","last_became_active":"2023-06-09T04:36:12.845198+0000","last_became_peered":"2023-06-09T04:36:12.845198+0000","last_unstale":"2023-06-09T04:36:14.378827+0000","last_undegraded":"2023-06-09T04:36:14.378827+0000","last_fullsized":"2023-06-09T04:36:14.378827+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:27:28.934134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962195+0000","last_change":"2023-06-09T04:36:12.440340+0000","last_active":"2023-06-09T04:36:13.962195+0000","last_peered":"2023-06-09T04:36:13.962195+0000","last_clean":"2023-06-09T04:36:13.962195+0000","last_became_active":"2023-06-09T04:36:12.440048+0000","last_became_peered":"2023-06-09T04:36:12.440048+0000","last_unstale":"2023-06-09T04:36:13.962195+0000","last_undegraded":"2023-06-09T04:36:13.962195+0000","last_fullsized":"2023-06-09T04:36:13.962195+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:42:42.219991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378806+0000","last_change":"2023-06-09T04:36:12.843376+0000","last_active":"2023-06-09T04:36:14.378806+0000","last_peered":"2023-06-09T04:36:14.378806+0000","last_clean":"2023-06-09T04:36:14.378806+0000","last_became_active":"2023-06-09T04:36:12.843097+0000","last_became_peered":"2023-06-09T04:36:12.843097+0000","last_unstale":"2023-06-09T04:36:14.378806+0000","last_undegraded":"2023-06-09T04:36:14.378806+0000","last_fullsized":"2023-06-09T04:36:14.378806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:29:03.620214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962145+0000","last_change":"2023-06-09T04:36:12.451004+0000","last_active":"2023-06-09T04:36:13.962145+0000","last_peered":"2023-06-09T04:36:13.962145+0000","last_clean":"2023-06-09T04:36:13.962145+0000","last_became_active":"2023-06-09T04:36:12.450718+0000","last_became_peered":"2023-06-09T04:36:12.450718+0000","last_unstale":"2023-06-09T04:36:13.962145+0000","last_undegraded":"2023-06-09T04:36:13.962145+0000","last_fullsized":"2023-06-09T04:36:13.962145+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:59:25.596099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961982+0000","last_change":"2023-06-09T04:36:12.840950+0000","last_active":"2023-06-09T04:36:13.961982+0000","last_peered":"2023-06-09T04:36:13.961982+0000","last_clean":"2023-06-09T04:36:13.961982+0000","last_became_active":"2023-06-09T04:36:12.840622+0000","last_became_peered":"2023-06-09T04:36:12.840622+0000","last_unstale":"2023-06-09T04:36:13.961982+0000","last_undegraded":"2023-06-09T04:36:13.961982+0000","last_fullsized":"2023-06-09T04:36:13.961982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:20:02.687890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378463+0000","last_change":"2023-06-09T04:36:12.444910+0000","last_active":"2023-06-09T04:36:14.378463+0000","last_peered":"2023-06-09T04:36:14.378463+0000","last_clean":"2023-06-09T04:36:14.378463+0000","last_became_active":"2023-06-09T04:36:12.444645+0000","last_became_peered":"2023-06-09T04:36:12.444645+0000","last_unstale":"2023-06-09T04:36:14.378463+0000","last_undegraded":"2023-06-09T04:36:14.378463+0000","last_fullsized":"2023-06-09T04:36:14.378463+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:05:07.148576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839011+0000","last_change":"2023-06-09T04:36:12.855925+0000","last_active":"2023-06-09T04:36:13.839011+0000","last_peered":"2023-06-09T04:36:13.839011+0000","last_clean":"2023-06-09T04:36:13.839011+0000","last_became_active":"2023-06-09T04:36:12.847191+0000","last_became_peered":"2023-06-09T04:36:12.847191+0000","last_unstale":"2023-06-09T04:36:13.839011+0000","last_undegraded":"2023-06-09T04:36:13.839011+0000","last_fullsized":"2023-06-09T04:36:13.839011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:09:30.979823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.836964+0000","last_change":"2023-06-09T04:36:12.445941+0000","last_active":"2023-06-09T04:36:13.836964+0000","last_peered":"2023-06-09T04:36:13.836964+0000","last_clean":"2023-06-09T04:36:13.836964+0000","last_became_active":"2023-06-09T04:36:12.445651+0000","last_became_peered":"2023-06-09T04:36:12.445651+0000","last_unstale":"2023-06-09T04:36:13.836964+0000","last_undegraded":"2023-06-09T04:36:13.836964+0000","last_fullsized":"2023-06-09T04:36:13.836964+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:41:56.097420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961933+0000","last_change":"2023-06-09T04:36:12.843164+0000","last_active":"2023-06-09T04:36:13.961933+0000","last_peered":"2023-06-09T04:36:13.961933+0000","last_clean":"2023-06-09T04:36:13.961933+0000","last_became_active":"2023-06-09T04:36:12.842079+0000","last_became_peered":"2023-06-09T04:36:12.842079+0000","last_unstale":"2023-06-09T04:36:13.961933+0000","last_undegraded":"2023-06-09T04:36:13.961933+0000","last_fullsized":"2023-06-09T04:36:13.961933+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:48:33.479326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839046+0000","last_change":"2023-06-09T04:36:12.448496+0000","last_active":"2023-06-09T04:36:13.839046+0000","last_peered":"2023-06-09T04:36:13.839046+0000","last_clean":"2023-06-09T04:36:13.839046+0000","last_became_active":"2023-06-09T04:36:12.448254+0000","last_became_peered":"2023-06-09T04:36:12.448254+0000","last_unstale":"2023-06-09T04:36:13.839046+0000","last_undegraded":"2023-06-09T04:36:13.839046+0000","last_fullsized":"2023-06-09T04:36:13.839046+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:54:26.599836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962062+0000","last_change":"2023-06-09T04:36:12.848874+0000","last_active":"2023-06-09T04:36:13.962062+0000","last_peered":"2023-06-09T04:36:13.962062+0000","last_clean":"2023-06-09T04:36:13.962062+0000","last_became_active":"2023-06-09T04:36:12.848591+0000","last_became_peered":"2023-06-09T04:36:12.848591+0000","last_unstale":"2023-06-09T04:36:13.962062+0000","last_undegraded":"2023-06-09T04:36:13.962062+0000","last_fullsized":"2023-06-09T04:36:13.962062+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:10:58.909090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840575+0000","last_change":"2023-06-09T04:36:12.448102+0000","last_active":"2023-06-09T04:36:13.840575+0000","last_peered":"2023-06-09T04:36:13.840575+0000","last_clean":"2023-06-09T04:36:13.840575+0000","last_became_active":"2023-06-09T04:36:12.447794+0000","last_became_peered":"2023-06-09T04:36:12.447794+0000","last_unstale":"2023-06-09T04:36:13.840575+0000","last_undegraded":"2023-06-09T04:36:13.840575+0000","last_fullsized":"2023-06-09T04:36:13.840575+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:33:32.158210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840425+0000","last_change":"2023-06-09T04:36:12.853064+0000","last_active":"2023-06-09T04:36:13.840425+0000","last_peered":"2023-06-09T04:36:13.840425+0000","last_clean":"2023-06-09T04:36:13.840425+0000","last_became_active":"2023-06-09T04:36:12.852757+0000","last_became_peered":"2023-06-09T04:36:12.852757+0000","last_unstale":"2023-06-09T04:36:13.840425+0000","last_undegraded":"2023-06-09T04:36:13.840425+0000","last_fullsized":"2023-06-09T04:36:13.840425+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:20:04.167346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378638+0000","last_change":"2023-06-09T04:36:12.447550+0000","last_active":"2023-06-09T04:36:14.378638+0000","last_peered":"2023-06-09T04:36:14.378638+0000","last_clean":"2023-06-09T04:36:14.378638+0000","last_became_active":"2023-06-09T04:36:12.447303+0000","last_became_peered":"2023-06-09T04:36:12.447303+0000","last_unstale":"2023-06-09T04:36:14.378638+0000","last_undegraded":"2023-06-09T04:36:14.378638+0000","last_fullsized":"2023-06-09T04:36:14.378638+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:52:14.050652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378745+0000","last_change":"2023-06-09T04:36:12.848115+0000","last_active":"2023-06-09T04:36:14.378745+0000","last_peered":"2023-06-09T04:36:14.378745+0000","last_clean":"2023-06-09T04:36:14.378745+0000","last_became_active":"2023-06-09T04:36:12.847679+0000","last_became_peered":"2023-06-09T04:36:12.847679+0000","last_unstale":"2023-06-09T04:36:14.378745+0000","last_undegraded":"2023-06-09T04:36:14.378745+0000","last_fullsized":"2023-06-09T04:36:14.378745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:02.522846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379468+0000","last_change":"2023-06-09T04:36:12.440911+0000","last_active":"2023-06-09T04:36:14.379468+0000","last_peered":"2023-06-09T04:36:14.379468+0000","last_clean":"2023-06-09T04:36:14.379468+0000","last_became_active":"2023-06-09T04:36:12.440648+0000","last_became_peered":"2023-06-09T04:36:12.440648+0000","last_unstale":"2023-06-09T04:36:14.379468+0000","last_undegraded":"2023-06-09T04:36:14.379468+0000","last_fullsized":"2023-06-09T04:36:14.379468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:15.950827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379636+0000","last_change":"2023-06-09T04:36:12.841287+0000","last_active":"2023-06-09T04:36:14.379636+0000","last_peered":"2023-06-09T04:36:14.379636+0000","last_clean":"2023-06-09T04:36:14.379636+0000","last_became_active":"2023-06-09T04:36:12.841013+0000","last_became_peered":"2023-06-09T04:36:12.841013+0000","last_unstale":"2023-06-09T04:36:14.379636+0000","last_undegraded":"2023-06-09T04:36:14.379636+0000","last_fullsized":"2023-06-09T04:36:14.379636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:19:37.751036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378694+0000","last_change":"2023-06-09T04:36:12.449225+0000","last_active":"2023-06-09T04:36:14.378694+0000","last_peered":"2023-06-09T04:36:14.378694+0000","last_clean":"2023-06-09T04:36:14.378694+0000","last_became_active":"2023-06-09T04:36:12.448936+0000","last_became_peered":"2023-06-09T04:36:12.448936+0000","last_unstale":"2023-06-09T04:36:14.378694+0000","last_undegraded":"2023-06-09T04:36:14.378694+0000","last_fullsized":"2023-06-09T04:36:14.378694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:48:32.477167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838758+0000","last_change":"2023-06-09T04:36:12.856939+0000","last_active":"2023-06-09T04:36:13.838758+0000","last_peered":"2023-06-09T04:36:13.838758+0000","last_clean":"2023-06-09T04:36:13.838758+0000","last_became_active":"2023-06-09T04:36:12.856146+0000","last_became_peered":"2023-06-09T04:36:12.856146+0000","last_unstale":"2023-06-09T04:36:13.838758+0000","last_undegraded":"2023-06-09T04:36:13.838758+0000","last_fullsized":"2023-06-09T04:36:13.838758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:00:17.968727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379245+0000","last_change":"2023-06-09T04:36:12.442518+0000","last_active":"2023-06-09T04:36:14.379245+0000","last_peered":"2023-06-09T04:36:14.379245+0000","last_clean":"2023-06-09T04:36:14.379245+0000","last_became_active":"2023-06-09T04:36:12.442212+0000","last_became_peered":"2023-06-09T04:36:12.442212+0000","last_unstale":"2023-06-09T04:36:14.379245+0000","last_undegraded":"2023-06-09T04:36:14.379245+0000","last_fullsized":"2023-06-09T04:36:14.379245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:39.057952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378473+0000","last_change":"2023-06-09T04:36:12.847386+0000","last_active":"2023-06-09T04:36:14.378473+0000","last_peered":"2023-06-09T04:36:14.378473+0000","last_clean":"2023-06-09T04:36:14.378473+0000","last_became_active":"2023-06-09T04:36:12.847080+0000","last_became_peered":"2023-06-09T04:36:12.847080+0000","last_unstale":"2023-06-09T04:36:14.378473+0000","last_undegraded":"2023-06-09T04:36:14.378473+0000","last_fullsized":"2023-06-09T04:36:14.378473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:45:25.433541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379224+0000","last_change":"2023-06-09T04:36:12.444092+0000","last_active":"2023-06-09T04:36:14.379224+0000","last_peered":"2023-06-09T04:36:14.379224+0000","last_clean":"2023-06-09T04:36:14.379224+0000","last_became_active":"2023-06-09T04:36:12.443813+0000","last_became_peered":"2023-06-09T04:36:12.443813+0000","last_unstale":"2023-06-09T04:36:14.379224+0000","last_undegraded":"2023-06-09T04:36:14.379224+0000","last_fullsized":"2023-06-09T04:36:14.379224+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T10:47:46.014477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840111+0000","last_change":"2023-06-09T04:36:12.849800+0000","last_active":"2023-06-09T04:36:13.840111+0000","last_peered":"2023-06-09T04:36:13.840111+0000","last_clean":"2023-06-09T04:36:13.840111+0000","last_became_active":"2023-06-09T04:36:12.849510+0000","last_became_peered":"2023-06-09T04:36:12.849510+0000","last_unstale":"2023-06-09T04:36:13.840111+0000","last_undegraded":"2023-06-09T04:36:13.840111+0000","last_fullsized":"2023-06-09T04:36:13.840111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:28:28.434740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839298+0000","last_change":"2023-06-09T04:36:12.444683+0000","last_active":"2023-06-09T04:36:13.839298+0000","last_peered":"2023-06-09T04:36:13.839298+0000","last_clean":"2023-06-09T04:36:13.839298+0000","last_became_active":"2023-06-09T04:36:12.444383+0000","last_became_peered":"2023-06-09T04:36:12.444383+0000","last_unstale":"2023-06-09T04:36:13.839298+0000","last_undegraded":"2023-06-09T04:36:13.839298+0000","last_fullsized":"2023-06-09T04:36:13.839298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:52:14.050652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962028+0000","last_change":"2023-06-09T04:36:12.856814+0000","last_active":"2023-06-09T04:36:13.962028+0000","last_peered":"2023-06-09T04:36:13.962028+0000","last_clean":"2023-06-09T04:36:13.962028+0000","last_became_active":"2023-06-09T04:36:12.856618+0000","last_became_peered":"2023-06-09T04:36:12.856618+0000","last_unstale":"2023-06-09T04:36:13.962028+0000","last_undegraded":"2023-06-09T04:36:13.962028+0000","last_fullsized":"2023-06-09T04:36:13.962028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:53:14.997329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378335+0000","last_change":"2023-06-09T04:36:12.442889+0000","last_active":"2023-06-09T04:36:14.378335+0000","last_peered":"2023-06-09T04:36:14.378335+0000","last_clean":"2023-06-09T04:36:14.378335+0000","last_became_active":"2023-06-09T04:36:12.442588+0000","last_became_peered":"2023-06-09T04:36:12.442588+0000","last_unstale":"2023-06-09T04:36:14.378335+0000","last_undegraded":"2023-06-09T04:36:14.378335+0000","last_fullsized":"2023-06-09T04:36:14.378335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:12:39.346840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378627+0000","last_change":"2023-06-09T04:36:12.837435+0000","last_active":"2023-06-09T04:36:14.378627+0000","last_peered":"2023-06-09T04:36:14.378627+0000","last_clean":"2023-06-09T04:36:14.378627+0000","last_became_active":"2023-06-09T04:36:12.837129+0000","last_became_peered":"2023-06-09T04:36:12.837129+0000","last_unstale":"2023-06-09T04:36:14.378627+0000","last_undegraded":"2023-06-09T04:36:14.378627+0000","last_fullsized":"2023-06-09T04:36:14.378627+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:08:35.982569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378340+0000","last_change":"2023-06-09T04:36:12.445127+0000","last_active":"2023-06-09T04:36:14.378340+0000","last_peered":"2023-06-09T04:36:14.378340+0000","last_clean":"2023-06-09T04:36:14.378340+0000","last_became_active":"2023-06-09T04:36:12.442493+0000","last_became_peered":"2023-06-09T04:36:12.442493+0000","last_unstale":"2023-06-09T04:36:14.378340+0000","last_undegraded":"2023-06-09T04:36:14.378340+0000","last_fullsized":"2023-06-09T04:36:14.378340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:29.777239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378642+0000","last_change":"2023-06-09T04:35:12.072943+0000","last_active":"2023-06-09T04:36:14.378642+0000","last_peered":"2023-06-09T04:36:14.378642+0000","last_clean":"2023-06-09T04:36:14.378642+0000","last_became_active":"2023-06-09T04:35:12.072623+0000","last_became_peered":"2023-06-09T04:35:12.072623+0000","last_unstale":"2023-06-09T04:36:14.378642+0000","last_undegraded":"2023-06-09T04:36:14.378642+0000","last_fullsized":"2023-06-09T04:36:14.378642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:13:01.027580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.962039+0000","last_change":"2023-06-09T04:36:12.851869+0000","last_active":"2023-06-09T04:36:13.962039+0000","last_peered":"2023-06-09T04:36:13.962039+0000","last_clean":"2023-06-09T04:36:13.962039+0000","last_became_active":"2023-06-09T04:36:12.851709+0000","last_became_peered":"2023-06-09T04:36:12.851709+0000","last_unstale":"2023-06-09T04:36:13.962039+0000","last_undegraded":"2023-06-09T04:36:13.962039+0000","last_fullsized":"2023-06-09T04:36:13.962039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:53:35.218311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378739+0000","last_change":"2023-06-09T04:36:12.445020+0000","last_active":"2023-06-09T04:36:14.378739+0000","last_peered":"2023-06-09T04:36:14.378739+0000","last_clean":"2023-06-09T04:36:14.378739+0000","last_became_active":"2023-06-09T04:36:12.444745+0000","last_became_peered":"2023-06-09T04:36:12.444745+0000","last_unstale":"2023-06-09T04:36:14.378739+0000","last_undegraded":"2023-06-09T04:36:14.378739+0000","last_fullsized":"2023-06-09T04:36:14.378739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:19.228681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840353+0000","last_change":"2023-06-09T04:35:12.002754+0000","last_active":"2023-06-09T04:36:13.840353+0000","last_peered":"2023-06-09T04:36:13.840353+0000","last_clean":"2023-06-09T04:36:13.840353+0000","last_became_active":"2023-06-09T04:35:12.002471+0000","last_became_peered":"2023-06-09T04:35:12.002471+0000","last_unstale":"2023-06-09T04:36:13.840353+0000","last_undegraded":"2023-06-09T04:36:13.840353+0000","last_fullsized":"2023-06-09T04:36:13.840353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:44.294575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839107+0000","last_change":"2023-06-09T04:36:12.854560+0000","last_active":"2023-06-09T04:36:13.839107+0000","last_peered":"2023-06-09T04:36:13.839107+0000","last_clean":"2023-06-09T04:36:13.839107+0000","last_became_active":"2023-06-09T04:36:12.854328+0000","last_became_peered":"2023-06-09T04:36:12.854328+0000","last_unstale":"2023-06-09T04:36:13.839107+0000","last_undegraded":"2023-06-09T04:36:13.839107+0000","last_fullsized":"2023-06-09T04:36:13.839107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:28:28.434740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379568+0000","last_change":"2023-06-09T04:36:12.440160+0000","last_active":"2023-06-09T04:36:14.379568+0000","last_peered":"2023-06-09T04:36:14.379568+0000","last_clean":"2023-06-09T04:36:14.379568+0000","last_became_active":"2023-06-09T04:36:12.439837+0000","last_became_peered":"2023-06-09T04:36:12.439837+0000","last_unstale":"2023-06-09T04:36:14.379568+0000","last_undegraded":"2023-06-09T04:36:14.379568+0000","last_fullsized":"2023-06-09T04:36:14.379568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:27:27.485483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.379039+0000","last_change":"2023-06-09T04:35:12.071356+0000","last_active":"2023-06-09T04:36:14.379039+0000","last_peered":"2023-06-09T04:36:14.379039+0000","last_clean":"2023-06-09T04:36:14.379039+0000","last_became_active":"2023-06-09T04:35:12.070997+0000","last_became_peered":"2023-06-09T04:35:12.070997+0000","last_unstale":"2023-06-09T04:36:14.379039+0000","last_undegraded":"2023-06-09T04:36:14.379039+0000","last_fullsized":"2023-06-09T04:36:14.379039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:58:21.528352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840498+0000","last_change":"2023-06-09T04:36:12.845301+0000","last_active":"2023-06-09T04:36:13.840498+0000","last_peered":"2023-06-09T04:36:13.840498+0000","last_clean":"2023-06-09T04:36:13.840498+0000","last_became_active":"2023-06-09T04:36:12.845013+0000","last_became_peered":"2023-06-09T04:36:12.845013+0000","last_unstale":"2023-06-09T04:36:13.840498+0000","last_undegraded":"2023-06-09T04:36:13.840498+0000","last_fullsized":"2023-06-09T04:36:13.840498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:03:41.465966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839172+0000","last_change":"2023-06-09T04:36:12.447828+0000","last_active":"2023-06-09T04:36:13.839172+0000","last_peered":"2023-06-09T04:36:13.839172+0000","last_clean":"2023-06-09T04:36:13.839172+0000","last_became_active":"2023-06-09T04:36:12.447358+0000","last_became_peered":"2023-06-09T04:36:12.447358+0000","last_unstale":"2023-06-09T04:36:13.839172+0000","last_undegraded":"2023-06-09T04:36:13.839172+0000","last_fullsized":"2023-06-09T04:36:13.839172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:09:08.126895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839951+0000","last_change":"2023-06-09T04:35:11.999027+0000","last_active":"2023-06-09T04:36:13.839951+0000","last_peered":"2023-06-09T04:36:13.839951+0000","last_clean":"2023-06-09T04:36:13.839951+0000","last_became_active":"2023-06-09T04:35:11.998723+0000","last_became_peered":"2023-06-09T04:35:11.998723+0000","last_unstale":"2023-06-09T04:36:13.839951+0000","last_undegraded":"2023-06-09T04:36:13.839951+0000","last_fullsized":"2023-06-09T04:36:13.839951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:40:53.096806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838530+0000","last_change":"2023-06-09T04:35:06.996716+0000","last_active":"2023-06-09T04:36:13.838530+0000","last_peered":"2023-06-09T04:36:13.838530+0000","last_clean":"2023-06-09T04:36:13.838530+0000","last_became_active":"2023-06-09T04:35:06.995565+0000","last_became_peered":"2023-06-09T04:35:06.995565+0000","last_unstale":"2023-06-09T04:36:13.838530+0000","last_undegraded":"2023-06-09T04:36:13.838530+0000","last_fullsized":"2023-06-09T04:36:13.838530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:05.906836+0000","last_clean_scrub_stamp":"2023-06-09T04:35:05.906836+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:36:55.796586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378540+0000","last_change":"2023-06-09T04:36:12.851150+0000","last_active":"2023-06-09T04:36:14.378540+0000","last_peered":"2023-06-09T04:36:14.378540+0000","last_clean":"2023-06-09T04:36:14.378540+0000","last_became_active":"2023-06-09T04:36:12.850913+0000","last_became_peered":"2023-06-09T04:36:12.850913+0000","last_unstale":"2023-06-09T04:36:14.378540+0000","last_undegraded":"2023-06-09T04:36:14.378540+0000","last_fullsized":"2023-06-09T04:36:14.378540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T05:08:23.307929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.840609+0000","last_change":"2023-06-09T04:36:12.441599+0000","last_active":"2023-06-09T04:36:13.840609+0000","last_peered":"2023-06-09T04:36:13.840609+0000","last_clean":"2023-06-09T04:36:13.840609+0000","last_became_active":"2023-06-09T04:36:12.441324+0000","last_became_peered":"2023-06-09T04:36:12.441324+0000","last_unstale":"2023-06-09T04:36:13.840609+0000","last_undegraded":"2023-06-09T04:36:13.840609+0000","last_fullsized":"2023-06-09T04:36:13.840609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:36:04.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.838487+0000","last_change":"2023-06-09T04:35:12.002654+0000","last_active":"2023-06-09T04:36:13.838487+0000","last_peered":"2023-06-09T04:36:13.838487+0000","last_clean":"2023-06-09T04:36:13.838487+0000","last_became_active":"2023-06-09T04:35:12.002320+0000","last_became_peered":"2023-06-09T04:35:12.002320+0000","last_unstale":"2023-06-09T04:36:13.838487+0000","last_undegraded":"2023-06-09T04:36:13.838487+0000","last_fullsized":"2023-06-09T04:36:13.838487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:15:00.010109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378854+0000","last_change":"2023-06-09T04:36:12.850116+0000","last_active":"2023-06-09T04:36:14.378854+0000","last_peered":"2023-06-09T04:36:14.378854+0000","last_clean":"2023-06-09T04:36:14.378854+0000","last_became_active":"2023-06-09T04:36:12.849864+0000","last_became_peered":"2023-06-09T04:36:12.849864+0000","last_unstale":"2023-06-09T04:36:14.378854+0000","last_undegraded":"2023-06-09T04:36:14.378854+0000","last_fullsized":"2023-06-09T04:36:14.378854+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T08:53:35.218311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837009+0000","last_change":"2023-06-09T04:36:12.442755+0000","last_active":"2023-06-09T04:36:13.837009+0000","last_peered":"2023-06-09T04:36:13.837009+0000","last_clean":"2023-06-09T04:36:13.837009+0000","last_became_active":"2023-06-09T04:36:12.442501+0000","last_became_peered":"2023-06-09T04:36:12.442501+0000","last_unstale":"2023-06-09T04:36:13.837009+0000","last_undegraded":"2023-06-09T04:36:13.837009+0000","last_fullsized":"2023-06-09T04:36:13.837009+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:38:47.022200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839190+0000","last_change":"2023-06-09T04:35:12.071967+0000","last_active":"2023-06-09T04:36:13.839190+0000","last_peered":"2023-06-09T04:36:13.839190+0000","last_clean":"2023-06-09T04:36:13.839190+0000","last_became_active":"2023-06-09T04:35:12.071565+0000","last_became_peered":"2023-06-09T04:35:12.071565+0000","last_unstale":"2023-06-09T04:36:13.839190+0000","last_undegraded":"2023-06-09T04:36:13.839190+0000","last_fullsized":"2023-06-09T04:36:13.839190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T04:54:41.668992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378546+0000","last_change":"2023-06-09T04:36:12.846958+0000","last_active":"2023-06-09T04:36:14.378546+0000","last_peered":"2023-06-09T04:36:14.378546+0000","last_clean":"2023-06-09T04:36:14.378546+0000","last_became_active":"2023-06-09T04:36:12.846444+0000","last_became_peered":"2023-06-09T04:36:12.846444+0000","last_unstale":"2023-06-09T04:36:14.378546+0000","last_undegraded":"2023-06-09T04:36:14.378546+0000","last_fullsized":"2023-06-09T04:36:14.378546+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:20:04.167346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378352+0000","last_change":"2023-06-09T04:36:12.446815+0000","last_active":"2023-06-09T04:36:14.378352+0000","last_peered":"2023-06-09T04:36:14.378352+0000","last_clean":"2023-06-09T04:36:14.378352+0000","last_became_active":"2023-06-09T04:36:12.445288+0000","last_became_peered":"2023-06-09T04:36:12.445288+0000","last_unstale":"2023-06-09T04:36:14.378352+0000","last_undegraded":"2023-06-09T04:36:14.378352+0000","last_fullsized":"2023-06-09T04:36:14.378352+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T06:09:08.126895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:14.378867+0000","last_change":"2023-06-09T04:35:12.003438+0000","last_active":"2023-06-09T04:36:14.378867+0000","last_peered":"2023-06-09T04:36:14.378867+0000","last_clean":"2023-06-09T04:36:14.378867+0000","last_became_active":"2023-06-09T04:35:12.003155+0000","last_became_peered":"2023-06-09T04:35:12.003155+0000","last_unstale":"2023-06-09T04:36:14.378867+0000","last_undegraded":"2023-06-09T04:36:14.378867+0000","last_fullsized":"2023-06-09T04:36:14.378867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T09:09:57.465969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.839210+0000","last_change":"2023-06-09T04:36:12.856608+0000","last_active":"2023-06-09T04:36:13.839210+0000","last_peered":"2023-06-09T04:36:13.839210+0000","last_clean":"2023-06-09T04:36:13.839210+0000","last_became_active":"2023-06-09T04:36:12.856228+0000","last_became_peered":"2023-06-09T04:36:12.856228+0000","last_unstale":"2023-06-09T04:36:13.839210+0000","last_undegraded":"2023-06-09T04:36:13.839210+0000","last_fullsized":"2023-06-09T04:36:13.839210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:12.422445+0000","last_clean_scrub_stamp":"2023-06-09T04:36:12.422445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:20:04.167346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.961852+0000","last_change":"2023-06-09T04:36:12.442988+0000","last_active":"2023-06-09T04:36:13.961852+0000","last_peered":"2023-06-09T04:36:13.961852+0000","last_clean":"2023-06-09T04:36:13.961852+0000","last_became_active":"2023-06-09T04:36:12.441199+0000","last_became_peered":"2023-06-09T04:36:12.441199+0000","last_unstale":"2023-06-09T04:36:13.961852+0000","last_undegraded":"2023-06-09T04:36:13.961852+0000","last_fullsized":"2023-06-09T04:36:13.961852+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:36:11.420287+0000","last_clean_scrub_stamp":"2023-06-09T04:36:11.420287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T07:14:01.468543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-09T04:36:13.837650+0000","last_change":"2023-06-09T04:35:12.003198+0000","last_active":"2023-06-09T04:36:13.837650+0000","last_peered":"2023-06-09T04:36:13.837650+0000","last_clean":"2023-06-09T04:36:13.837650+0000","last_became_active":"2023-06-09T04:35:12.002905+0000","last_became_peered":"2023-06-09T04:35:12.002905+0000","last_unstale":"2023-06-09T04:36:13.837650+0000","last_undegraded":"2023-06-09T04:36:13.837650+0000","last_fullsized":"2023-06-09T04:36:13.837650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T04:35:10.979324+0000","last_clean_scrub_stamp":"2023-06-09T04:35:10.979324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:08:38.903536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":1044480,"data_stored":772053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":1.9670000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":8.4540000000000006,"5min":8.4540000000000006,"15min":8.4540000000000006},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45000000000000001}]},{"osd":5,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":4.3319999999999999,"5min":4.3319999999999999,"15min":4.3319999999999999},"last":0.432}]},{"osd":6,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.505},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":180633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.371},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.496},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58899999999999997}]},{"osd":7,"last update":"Fri Jun 9 04:36:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":173488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.312},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32400000000000001}]},{"osd":2,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.5}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.48499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":1015808,"data_stored":763131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":4.5990000000000002,"5min":4.5990000000000002,"15min":4.5990000000000002},"last":0.18099999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":5.9800000000000004,"5min":5.9800000000000004,"15min":5.9800000000000004},"last":0.24099999999999999}]},{"osd":2,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":6.0250000000000004,"5min":6.0250000000000004,"15min":6.0250000000000004},"last":0.20899999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.0590000000000002,"5min":5.0590000000000002,"15min":5.0590000000000002},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.673,"5min":4.673,"15min":4.673},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.0220000000000002,"5min":5.0220000000000002,"15min":5.0220000000000002},"last":0.34000000000000002}]},{"osd":4,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":5.9199999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.5}]},{"osd":5,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":5.9539999999999997,"5min":5.9539999999999997,"15min":5.9539999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.5659999999999998,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.6500000000000004,"5min":4.6500000000000004,"15min":4.6500000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":6.0030000000000001,"5min":6.0030000000000001,"15min":6.0030000000000001},"last":0.379}]},{"osd":7,"last update":"Fri Jun 9 04:36:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":3.6059999999999999,"5min":3.6059999999999999,"15min":3.6059999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.40400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":173468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.434,"5min":0.434,"15min":0.434},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.33200000000000002}]},{"osd":1,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.318},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.308}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":440,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":450560,"data_stored":181649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.307,"15min":0.307},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.254},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.32700000000000001}]},{"osd":1,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.29599999999999999}]},{"osd":2,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.316},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.34399999999999997}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.496}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":174002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.45400000000000001}]},{"osd":6,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Jun 9 04:36:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.67100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":180163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.40699999999999997}]},{"osd":6,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.45500000000000002}]},{"osd":7,"last update":"Fri Jun 9 04:36:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T04:36:17.350 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-09T04:36:17.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-06-09T04:36:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.507+0000 7f80a39d6700 1 Processor -- start 2023-06-09T04:36:17.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.508+0000 7f80a39d6700 1 -- start start 2023-06-09T04:36:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.508+0000 7f80a39d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cf440 0x7f809c15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.508+0000 7f80a39d6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f809c0cce60 0x7f809c06ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.508+0000 7f80a39d6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c15d040 con 0x7f809c069db0 2023-06-09T04:36:17.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.508+0000 7f80a39d6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c160ab0 con 0x7f809c0cce60 2023-06-09T04:36:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.508+0000 7f80a39d6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c15ca10 con 0x7f809c0cf440 2023-06-09T04:36:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1f73700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f809c0cce60 0x7f809c06ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1772700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f809c069db0 0x7f809c06a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38012/0 (socket says 172.21.15.92:38012) 2023-06-09T04:36:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1772700 1 -- 172.21.15.92:0/2400074480 learned_addr learned my addr 172.21.15.92:0/2400074480 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a0f71700 1 --2- 172.21.15.92:0/2400074480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cf440 0x7f809c15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1f73700 1 -- 172.21.15.92:0/2400074480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cf440 msgr2=0x7f809c15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:17.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1f73700 1 --2- 172.21.15.92:0/2400074480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cf440 0x7f809c15d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1f73700 1 -- 172.21.15.92:0/2400074480 >> v1:172.21.15.92:6789/0 conn(0x7f809c069db0 legacy=0x7f809c06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f8093fff700 1 -- 172.21.15.92:0/2400074480 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744436661 0 0) 0x7f809c15d040 con 0x7f809c069db0 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1f73700 1 -- 172.21.15.92:0/2400074480 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c160d50 con 0x7f809c0cce60 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f80a1f73700 1 --2- 172.21.15.92:0/2400074480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c0cce60 0x7f809c06ee20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8094009d90 tx=0x7f809400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588e47f467f44b9c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f8093fff700 1 -- 172.21.15.92:0/2400074480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400e070 con 0x7f809c0cce60 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f8093fff700 1 -- 172.21.15.92:0/2400074480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809400bbd0 con 0x7f809c0cce60 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.509+0000 7f8093fff700 1 -- 172.21.15.92:0/2400074480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094012430 con 0x7f809c0cce60 2023-06-09T04:36:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 -- 172.21.15.92:0/2400074480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c0cce60 msgr2=0x7f809c06ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2400074480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c0cce60 0x7f809c06ee20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8094009d90 tx=0x7f809400b330 comp rx=0 tx=0).stop 2023-06-09T04:36:17.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 -- 172.21.15.92:0/2400074480 shutdown_connections 2023-06-09T04:36:17.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2400074480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c0cce60 0x7f809c06ee20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2400074480 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cf440 0x7f809c15d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 -- 172.21.15.92:0/2400074480 >> 172.21.15.92:0/2400074480 conn(0x7f809c0cc380 msgr2=0x7f809c0cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:17.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 -- 172.21.15.92:0/2400074480 shutdown_connections 2023-06-09T04:36:17.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.510+0000 7f80a39d6700 1 -- 172.21.15.92:0/2400074480 wait complete. 2023-06-09T04:36:17.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.511+0000 7f80a39d6700 1 Processor -- start 2023-06-09T04:36:17.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.511+0000 7f80a39d6700 1 -- start start 2023-06-09T04:36:17.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a39d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c069db0 0x7f809c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a39d6700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cce60 0x7f809c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a39d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f809c0cf440 0x7f809c0b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a39d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0b21f0 con 0x7f809c069db0 2023-06-09T04:36:17.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a39d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0b15e0 con 0x7f809c0cf440 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a39d6700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0b18c0 con 0x7f809c0cce60 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c069db0 0x7f809c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c069db0 0x7f809c155f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39686/0 (socket says 172.21.15.92:39686) 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 -- 172.21.15.92:0/2617449503 learned_addr learned my addr 172.21.15.92:0/2617449503 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a0f71700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cce60 0x7f809c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1f73700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f809c0cf440 0x7f809c0b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 -- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cce60 msgr2=0x7f809c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cce60 0x7f809c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 -- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f809c0cf440 msgr2=0x7f809c0b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f809c0cf440 0x7f809c0b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 -- 172.21.15.92:0/2617449503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094009a70 con 0x7f809c069db0 2023-06-09T04:36:17.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a0f71700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cce60 0x7f809c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:17.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1f73700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f809c0cf440 0x7f809c0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:17.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a1772700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c069db0 0x7f809c155f00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f808c0011e0 tx=0x7f808c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f8091ffb700 1 -- 172.21.15.92:0/2617449503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c004c80 con 0x7f809c069db0 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f8091ffb700 1 -- 172.21.15.92:0/2617449503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c004e00 con 0x7f809c069db0 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c0b1ba0 con 0x7f809c069db0 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.512+0000 7f8091ffb700 1 -- 172.21.15.92:0/2617449503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c007310 con 0x7f809c069db0 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.513+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c0b0840 con 0x7f809c069db0 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.514+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c06be50 con 0x7f809c069db0 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.514+0000 7f8091ffb700 1 -- 172.21.15.92:0/2617449503 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f808c007470 con 0x7f809c069db0 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.514+0000 7f8091ffb700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f808006ace0 0x7f808006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.515+0000 7f8091ffb700 1 -- 172.21.15.92:0/2617449503 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f808c089200 con 0x7f809c069db0 2023-06-09T04:36:17.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.515+0000 7f80a0f71700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f808006ace0 0x7f808006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.515+0000 7f80a0f71700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f808006ace0 0x7f808006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f809c0b4ce0 tx=0x7f8098008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.519+0000 7f8091ffb700 1 -- 172.21.15.92:0/2617449503 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f808c0542b0 con 0x7f809c069db0 2023-06-09T04:36:17.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.783+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f809c04e8e0 con 0x7f809c069db0 2023-06-09T04:36:17.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.788+0000 7f8091ffb700 1 -- 172.21.15.92:0/2617449503 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f808c0578d0 con 0x7f809c069db0 2023-06-09T04:36:17.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f808006ace0 msgr2=0x7f808006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f808006ace0 0x7f808006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f809c0b4ce0 tx=0x7f8098008040 comp rx=0 tx=0).stop 2023-06-09T04:36:17.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c069db0 msgr2=0x7f809c155f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c069db0 0x7f809c155f00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f808c0011e0 tx=0x7f808c00cae0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 shutdown_connections 2023-06-09T04:36:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f808006ace0 0x7f808006d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f809c0cf440 0x7f809c0b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f809c069db0 0x7f809c155f00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 --2- 172.21.15.92:0/2617449503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809c0cce60 0x7f809c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 >> 172.21.15.92:0/2617449503 conn(0x7f809c0cc380 msgr2=0x7f809c071390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 shutdown_connections 2023-06-09T04:36:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.791+0000 7f80a39d6700 1 -- 172.21.15.92:0/2617449503 wait complete. 2023-06-09T04:36:17.794 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-06-09T04:36:17.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T04:36:17.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.924+0000 7f342acdc700 1 Processor -- start 2023-06-09T04:36:17.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.924+0000 7f342acdc700 1 -- start start 2023-06-09T04:36:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.924+0000 7f342acdc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34240ae630 0x7f34240aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.924+0000 7f342acdc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240cd7e0 0x7f34240cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.924+0000 7f342acdc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34240adde0 con 0x7f34240b1360 2023-06-09T04:36:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.924+0000 7f342acdc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240ae270 con 0x7f34240ae630 2023-06-09T04:36:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.924+0000 7f342acdc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240af0f0 con 0x7f34240cd7e0 2023-06-09T04:36:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3428a78700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f34240b1360 0x7f34240b1730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38026/0 (socket says 172.21.15.92:38026) 2023-06-09T04:36:17.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3428a78700 1 -- 172.21.15.92:0/3168774775 learned_addr learned my addr 172.21.15.92:0/3168774775 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3423fff700 1 --2- 172.21.15.92:0/3168774775 >> v2:172.21.15.123:3300/0 conn(0x7f34240ae630 0x7f34240aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3429279700 1 --2- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240cd7e0 0x7f34240cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f34237fe700 1 -- 172.21.15.92:0/3168774775 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712513954 0 0) 0x7f34240adde0 con 0x7f34240b1360 2023-06-09T04:36:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f34237fe700 1 -- 172.21.15.92:0/3168774775 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34140035b0 con 0x7f34240b1360 2023-06-09T04:36:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3423fff700 1 -- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240cd7e0 msgr2=0x7f34240cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3423fff700 1 --2- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240cd7e0 0x7f34240cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3423fff700 1 -- 172.21.15.92:0/3168774775 >> v1:172.21.15.92:6789/0 conn(0x7f34240b1360 legacy=0x7f34240b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3423fff700 1 -- 172.21.15.92:0/3168774775 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240ad4b0 con 0x7f34240ae630 2023-06-09T04:36:17.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3429279700 1 --2- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240cd7e0 0x7f34240cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:36:17.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f3423fff700 1 --2- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240ae630 0x7f34240aeaa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3410009840 tx=0x7f3410009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41621d2a1b4a35ce server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.925+0000 7f34237fe700 1 -- 172.21.15.92:0/3168774775 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011830 con 0x7f34240ae630 2023-06-09T04:36:17.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f34237fe700 1 -- 172.21.15.92:0/3168774775 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410011990 con 0x7f34240ae630 2023-06-09T04:36:17.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f34237fe700 1 -- 172.21.15.92:0/3168774775 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410011d20 con 0x7f34240ae630 2023-06-09T04:36:17.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 -- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240ae630 msgr2=0x7f34240aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 --2- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240ae630 0x7f34240aeaa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3410009840 tx=0x7f3410009b50 comp rx=0 tx=0).stop 2023-06-09T04:36:17.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 -- 172.21.15.92:0/3168774775 shutdown_connections 2023-06-09T04:36:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 --2- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240cd7e0 0x7f34240cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 --2- 172.21.15.92:0/3168774775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240ae630 0x7f34240aeaa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 -- 172.21.15.92:0/3168774775 >> 172.21.15.92:0/3168774775 conn(0x7f3424098c10 msgr2=0x7f34240cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 -- 172.21.15.92:0/3168774775 shutdown_connections 2023-06-09T04:36:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.926+0000 7f342acdc700 1 -- 172.21.15.92:0/3168774775 wait complete. 2023-06-09T04:36:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.927+0000 7f342acdc700 1 Processor -- start 2023-06-09T04:36:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.927+0000 7f342acdc700 1 -- start start 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f342acdc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240ae630 0x7f34241f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f342acdc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34240b1360 0x7f3424147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f342acdc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240cd7e0 0x7f34241ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f342acdc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240c3600 con 0x7f34240b1360 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f342acdc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34241ef200 con 0x7f34240cd7e0 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f342acdc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34241ef4e0 con 0x7f34240ae630 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f3423fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34240b1360 0x7f3424147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f3423fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34240b1360 0x7f3424147e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39702/0 (socket says 172.21.15.92:39702) 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.928+0000 7f3423fff700 1 -- 172.21.15.92:0/1203778317 learned_addr learned my addr 172.21.15.92:0/1203778317 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3429279700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240cd7e0 0x7f34241ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3428a78700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240ae630 0x7f34241f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3423fff700 1 -- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240ae630 msgr2=0x7f34241f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3423fff700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240ae630 0x7f34241f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3423fff700 1 -- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240cd7e0 msgr2=0x7f34241ee9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3423fff700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240cd7e0 0x7f34241ee9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:17.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3423fff700 1 -- 172.21.15.92:0/1203778317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c009a70 con 0x7f34240b1360 2023-06-09T04:36:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3429279700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240cd7e0 0x7f34241ee9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3428a78700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240ae630 0x7f34241f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f3423fff700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34240b1360 0x7f3424147e00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f34100090b0 tx=0x7f3410004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f34217fa700 1 -- 172.21.15.92:0/1203778317 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410004590 con 0x7f34240b1360 2023-06-09T04:36:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f34217fa700 1 -- 172.21.15.92:0/1203778317 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341001f070 con 0x7f34240b1360 2023-06-09T04:36:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34100094a0 con 0x7f34240b1360 2023-06-09T04:36:17.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f34217fa700 1 -- 172.21.15.92:0/1203778317 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100038e0 con 0x7f34240b1360 2023-06-09T04:36:17.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.929+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34241eeee0 con 0x7f34240b1360 2023-06-09T04:36:17.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.930+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34241efa00 con 0x7f34240b1360 2023-06-09T04:36:17.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.930+0000 7f34217fa700 1 -- 172.21.15.92:0/1203778317 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3410003a40 con 0x7f34240b1360 2023-06-09T04:36:17.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.931+0000 7f34217fa700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f340806ace0 0x7f340806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:17.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.931+0000 7f34217fa700 1 -- 172.21.15.92:0/1203778317 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3410021070 con 0x7f34240b1360 2023-06-09T04:36:17.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.931+0000 7f3428a78700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f340806ace0 0x7f340806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:17.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.931+0000 7f3428a78700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f340806ace0 0x7f340806d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f34180010e0 tx=0x7f3418009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:17.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:17.935+0000 7f34217fa700 1 -- 172.21.15.92:0/1203778317 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3410055c30 con 0x7f34240b1360 2023-06-09T04:36:18.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.164+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34240740e0 con 0x7f34240b1360 2023-06-09T04:36:18.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.164+0000 7f34217fa700 1 -- 172.21.15.92:0/1203778317 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f3410059250 con 0x7f34240b1360 2023-06-09T04:36:18.166 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:18.166 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6834/2062617587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":2062617587},{"type":"v1","addr":"172.21.15.123:6834","nonce":2062617587}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/1352099741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6833","nonce":1352099741},{"type":"v1","addr":"172.21.15.123:6835","nonce":1352099741}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2293053570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2293053570},{"type":"v1","addr":"172.21.15.92:6836","nonce":2293053570}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T04:36:12.823788+0000","modified":"2023-06-09T04:36:15.838310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/637518437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":637518437},{"type":"v1","addr":"172.21.15.92:6837","nonce":637518437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-09T04:36:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f340806ace0 msgr2=0x7f340806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f340806ace0 0x7f340806d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f34180010e0 tx=0x7f3418009040 comp rx=0 tx=0).stop 2023-06-09T04:36:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34240b1360 msgr2=0x7f3424147e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34240b1360 0x7f3424147e00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f34100090b0 tx=0x7f3410004750 comp rx=0 tx=0).stop 2023-06-09T04:36:18.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 shutdown_connections 2023-06-09T04:36:18.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f340806ace0 0x7f340806d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34240cd7e0 0x7f34241ee9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f34240ae630 0x7f34241f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 --2- 172.21.15.92:0/1203778317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34240b1360 0x7f3424147e00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 >> 172.21.15.92:0/1203778317 conn(0x7f3424098c10 msgr2=0x7f34240cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 shutdown_connections 2023-06-09T04:36:18.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.166+0000 7f342acdc700 1 -- 172.21.15.92:0/1203778317 wait complete. 2023-06-09T04:36:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2023-06-09T04:36:18.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-09T04:36:12.823788+0000', 'modified': '2023-06-09T04:36:15.838310+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5127': {'gid': 5127, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/637518437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 637518437}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 637518437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-09T04:36:18.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-09T04:36:18.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-06-09T04:36:18.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.703+0000 7f3acd1b5700 1 Processor -- start 2023-06-09T04:36:18.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.703+0000 7f3acd1b5700 1 -- start start 2023-06-09T04:36:18.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3acd1b5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ac815c7d0 0x7f3ac80ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3acd1b5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80ce730 con 0x7f3ac80cf010 2023-06-09T04:36:18.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3acd1b5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac806aaf0 con 0x7f3ac80ac400 2023-06-09T04:36:18.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3acd1b5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac815cba0 con 0x7f3ac815c7d0 2023-06-09T04:36:18.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ac815c7d0 0x7f3ac80ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac759e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3ac80cf010 0x7f3ac806e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38042/0 (socket says 172.21.15.92:38042) 2023-06-09T04:36:18.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac759e700 1 -- 172.21.15.92:0/3570693245 learned_addr learned my addr 172.21.15.92:0/3570693245 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:18.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/3570693245 >> v1:172.21.15.123:6789/0 conn(0x7f3ac80ac400 legacy=0x7f3ac80ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/3570693245 >> v1:172.21.15.92:6789/0 conn(0x7f3ac80cf010 legacy=0x7f3ac806e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac5d9b700 1 -- 172.21.15.92:0/3570693245 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178980243 0 0) 0x7f3ac0001490 con 0x7f3ac80cf010 2023-06-09T04:36:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/3570693245 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8069ee0 con 0x7f3ac815c7d0 2023-06-09T04:36:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/3570693245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac815c7d0 0x7f3ac80ce1f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009820 tx=0x7f3ab8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde2621c3ba8c84c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac5d9b700 1 -- 172.21.15.92:0/3570693245 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8005e30 con 0x7f3ac815c7d0 2023-06-09T04:36:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac5d9b700 1 -- 172.21.15.92:0/3570693245 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8004030 con 0x7f3ac815c7d0 2023-06-09T04:36:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.704+0000 7f3ac5d9b700 1 -- 172.21.15.92:0/3570693245 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80043c0 con 0x7f3ac815c7d0 2023-06-09T04:36:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.705+0000 7f3acd1b5700 1 -- 172.21.15.92:0/3570693245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac815c7d0 msgr2=0x7f3ac80ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.705+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/3570693245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac815c7d0 0x7f3ac80ce1f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009820 tx=0x7f3ab8009b30 comp rx=0 tx=0).stop 2023-06-09T04:36:18.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.705+0000 7f3acd1b5700 1 -- 172.21.15.92:0/3570693245 shutdown_connections 2023-06-09T04:36:18.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.705+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/3570693245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac815c7d0 0x7f3ac80ce1f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.705+0000 7f3acd1b5700 1 -- 172.21.15.92:0/3570693245 >> 172.21.15.92:0/3570693245 conn(0x7f3ac8098c10 msgr2=0x7f3ac80cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.705+0000 7f3acd1b5700 1 -- 172.21.15.92:0/3570693245 shutdown_connections 2023-06-09T04:36:18.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.705+0000 7f3acd1b5700 1 -- 172.21.15.92:0/3570693245 wait complete. 2023-06-09T04:36:18.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.706+0000 7f3acd1b5700 1 Processor -- start 2023-06-09T04:36:18.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.706+0000 7f3acd1b5700 1 -- start start 2023-06-09T04:36:18.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3acd1b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac80ac400 0x7f3ac80c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3acd1b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac80cf010 0x7f3ac814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3acd1b5700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac815c7d0 0x7f3ac80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3acd1b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80b4b70 con 0x7f3ac80ac400 2023-06-09T04:36:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3acd1b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac814e310 con 0x7f3ac80cf010 2023-06-09T04:36:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3acd1b5700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80b3ef0 con 0x7f3ac815c7d0 2023-06-09T04:36:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac80ac400 0x7f3ac80c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac80ac400 0x7f3ac80c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39712/0 (socket says 172.21.15.92:39712) 2023-06-09T04:36:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.707+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/1122717382 learned_addr learned my addr 172.21.15.92:0/1122717382 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac815c7d0 msgr2=0x7f3ac80b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac815c7d0 0x7f3ac80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac80cf010 msgr2=0x7f3ac814ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac659c700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac80cf010 0x7f3ac814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac80cf010 0x7f3ac814ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/1122717382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8009430 con 0x7f3ac80ac400 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac659c700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac80cf010 0x7f3ac814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac80ac400 0x7f3ac80c7bb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8000c00 tx=0x7f3ab8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3aaf7fe700 1 -- 172.21.15.92:0/1122717382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab801d630 con 0x7f3ac80ac400 2023-06-09T04:36:18.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3aaf7fe700 1 -- 172.21.15.92:0/1122717382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab801d7b0 con 0x7f3ac80ac400 2023-06-09T04:36:18.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac80b41d0 con 0x7f3ac80ac400 2023-06-09T04:36:18.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3aaf7fe700 1 -- 172.21.15.92:0/1122717382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab801db60 con 0x7f3ac80ac400 2023-06-09T04:36:18.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.708+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac80b4790 con 0x7f3ac80ac400 2023-06-09T04:36:18.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.709+0000 7f3aaf7fe700 1 -- 172.21.15.92:0/1122717382 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ab801dcc0 con 0x7f3ac80ac400 2023-06-09T04:36:18.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.710+0000 7f3aaf7fe700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3ab406ace0 0x7f3ab406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.710+0000 7f3aaf7fe700 1 -- 172.21.15.92:0/1122717382 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ab808aba0 con 0x7f3ac80ac400 2023-06-09T04:36:18.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.710+0000 7f3ac659c700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3ab406ace0 0x7f3ab406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.710+0000 7f3ac659c700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3ab406ace0 0x7f3ab406d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3abc006fd0 tx=0x7f3abc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.712+0000 7f3ad933ab80 1 Processor -- start 2023-06-09T04:36:18.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.712+0000 7f3ad933ab80 1 -- start start 2023-06-09T04:36:18.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0455310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04596c0 0x555df0459b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df035f3b0 con 0x555df04592f0 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df0360c60 con 0x555df04a2c70 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555df03b3570 con 0x555df04596c0 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0455310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0455310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39720/0 (socket says 172.21.15.92:39720) 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/634827694 learned_addr learned my addr 172.21.15.92:0/634827694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:18.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac659c700 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04596c0 0x555df0459b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac759e700 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04596c0 msgr2=0x555df0459b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04596c0 0x555df0459b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 msgr2=0x555df04a30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/634827694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df01dad90 con 0x555df04592f0 2023-06-09T04:36:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.713+0000 7f3ac759e700 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0455310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8061740 tx=0x7f3ab80596f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d006231bc5c0dff7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3aacff9700 1 -- 172.21.15.92:0/634827694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8061c70 con 0x555df04592f0 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3aacff9700 1 -- 172.21.15.92:0/634827694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8057cc0 con 0x555df04592f0 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3aacff9700 1 -- 172.21.15.92:0/634827694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8059010 con 0x555df04592f0 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ad933ab80 1 -- 172.21.15.92:0/634827694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 msgr2=0x555df0455310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0455310 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8061740 tx=0x7f3ab80596f0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ad933ab80 1 -- 172.21.15.92:0/634827694 shutdown_connections 2023-06-09T04:36:18.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0455310 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/634827694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04596c0 0x555df0459b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.714+0000 7f3ad933ab80 1 -- 172.21.15.92:0/634827694 >> 172.21.15.92:0/634827694 conn(0x555df0457ce0 msgr2=0x555df04a04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.715+0000 7f3ad933ab80 1 -- 172.21.15.92:0/634827694 shutdown_connections 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.715+0000 7f3ad933ab80 1 -- 172.21.15.92:0/634827694 wait complete. 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.715+0000 7f3ad933ab80 1 -- start start 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.715+0000 7f3ad933ab80 1 Processor -- start 2023-06-09T04:36:18.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0483490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04596c0 0x555df049edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df049f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df03aca00 con 0x555df04592f0 2023-06-09T04:36:18.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df034ca70 con 0x555df04596c0 2023-06-09T04:36:18.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555df034c770 con 0x555df04a2c70 2023-06-09T04:36:18.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ad933ab80 10 client.0 _ll_register_callbacks cb 0x555df034a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T04:36:18.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0483490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0483490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39726/0 (socket says 172.21.15.92:39726) 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/2091568996 learned_addr learned my addr 172.21.15.92:0/2091568996 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac759e700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df049f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac659c700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04596c0 0x555df049edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 msgr2=0x555df049f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df049f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04596c0 msgr2=0x555df049edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04596c0 0x555df049edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 -- 172.21.15.92:0/2091568996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac80b4790 con 0x555df04592f0 2023-06-09T04:36:18.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac659c700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04596c0 0x555df049edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:36:18.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0483490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8055e20 tx=0x7f3ab8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db080749ec106a8d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.716+0000 7f3aa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-06-09T04:36:18.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2091568996 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8061c70 con 0x555df04592f0 2023-06-09T04:36:18.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2091568996 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ab80617e0 con 0x555df04592f0 2023-06-09T04:36:18.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555df044d850 con 0x555df04592f0 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2091568996 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab80619c0 con 0x555df04592f0 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 10 client.5334 fetch_fsmap learned FSMap version 16 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x555df01c3c10 con 0x555df04592f0 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2091568996 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555df044d850 con 0x555df04592f0 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2091568996 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3ab805db40 con 0x555df04592f0 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 10 client.5334 fetch_fsmap finished waiting for FSMap version 16 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 10 client.5334 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-09T04:36:18.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 20 client.5334 populate_metadata read hostname 'smithi092' 2023-06-09T04:36:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df04b3410 0x555df04b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 4 client.5334 mds_command: new command op to 5127 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-09T04:36:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 --> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555df03a59e0 con 0x555df04b3410 2023-06-09T04:36:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ac659c700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df04b3410 0x555df04b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3aa5ffb700 10 client.5334 ms_handle_connect on v2:172.21.15.92:6835/637518437 2023-06-09T04:36:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.717+0000 7f3ac659c700 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df04b3410 0x555df04b37e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7923d8b3dc428489 server_cookie=253553b40b3ff77f in_seq=0 out_seq=0 2023-06-09T04:36:18.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2091568996 <== mds.0 v2:172.21.15.92:6835/637518437 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555df03a59e0 con 0x555df04b3410 2023-06-09T04:36:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3aa5ffb700 10 client.5334 handle_command_reply: tid=0 2023-06-09T04:36:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 client.5334 shutdown 2023-06-09T04:36:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 msgr2=0x555df0483490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0483490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8055e20 tx=0x7f3ab8004360 comp rx=0 tx=0).stop 2023-06-09T04:36:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 shutdown_connections 2023-06-09T04:36:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df049f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df04b3410 0x555df04b37e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04592f0 0x555df0483490 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2091568996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04596c0 0x555df049edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3aa5ffb700 0 client.5334 ms_handle_reset on v2:172.21.15.92:6835/637518437 2023-06-09T04:36:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 >> 172.21.15.92:0/2091568996 conn(0x555df0457ce0 msgr2=0x555df04a0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 shutdown_connections 2023-06-09T04:36:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2091568996 wait complete. 2023-06-09T04:36:18.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.718+0000 7f3ad933ab80 20 client.5334 trim_cache size 0 max 16384 2023-06-09T04:36:18.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 Processor -- start 2023-06-09T04:36:18.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 -- start start 2023-06-09T04:36:18.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04592f0 0x555df049c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df0457680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df0489100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df02308d0 con 0x555df04596c0 2023-06-09T04:36:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df0456910 con 0x555df04592f0 2023-06-09T04:36:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555df049c2d0 con 0x555df04a2c70 2023-06-09T04:36:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.736+0000 7f3ac659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df0457680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df0457680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39736/0 (socket says 172.21.15.92:39736) 2023-06-09T04:36:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 -- 172.21.15.92:0/3874197136 learned_addr learned my addr 172.21.15.92:0/3874197136 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:18.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 -- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 msgr2=0x555df0489100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac759e700 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df0489100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04592f0 0x555df049c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df0489100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 -- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04592f0 msgr2=0x555df049c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04592f0 0x555df049c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 -- 172.21.15.92:0/3874197136 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df04a1de0 con 0x555df04596c0 2023-06-09T04:36:18.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3ac659c700 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df0457680 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00a1c0 tx=0x7f3abc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5a0bb60406dc9a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3aad7fa700 1 -- 172.21.15.92:0/3874197136 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc025040 con 0x555df04596c0 2023-06-09T04:36:18.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3aad7fa700 1 -- 172.21.15.92:0/3874197136 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc026440 con 0x555df04596c0 2023-06-09T04:36:18.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.737+0000 7f3aad7fa700 1 -- 172.21.15.92:0/3874197136 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0269f0 con 0x555df04596c0 2023-06-09T04:36:18.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 -- 172.21.15.92:0/3874197136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 msgr2=0x555df0457680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df0457680 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00a1c0 tx=0x7f3abc011000 comp rx=0 tx=0).stop 2023-06-09T04:36:18.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 -- 172.21.15.92:0/3874197136 shutdown_connections 2023-06-09T04:36:18.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04a2c70 0x555df0489100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04592f0 0x555df049c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/3874197136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df0457680 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 -- 172.21.15.92:0/3874197136 >> 172.21.15.92:0/3874197136 conn(0x555df0457ce0 msgr2=0x555df044eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 -- 172.21.15.92:0/3874197136 shutdown_connections 2023-06-09T04:36:18.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 -- 172.21.15.92:0/3874197136 wait complete. 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 -- start start 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.738+0000 7f3ad933ab80 1 Processor -- start 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.739+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04592f0 0x555df0470280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df04af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df04558e0 con 0x555df04596c0 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df04a3b10 con 0x555df04a2c70 2023-06-09T04:36:18.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x555df0488cb0 con 0x555df04592f0 2023-06-09T04:36:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df04af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 10 client.0 _ll_register_callbacks cb 0x555df034a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T04:36:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df04af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39742/0 (socket says 172.21.15.92:39742) 2023-06-09T04:36:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 -- 172.21.15.92:0/2402076601 learned_addr learned my addr 172.21.15.92:0/2402076601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac759e700 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 -- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04592f0 msgr2=0x555df0470280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04592f0 0x555df0470280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04592f0 0x555df0470280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 -- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 msgr2=0x555df04a3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 -- 172.21.15.92:0/2402076601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc018040 con 0x555df04596c0 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ac659c700 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df04af1f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x555df0471090 tx=0x7f3abc004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e403ca23598c1323 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3aa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2402076601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc025070 con 0x555df04596c0 2023-06-09T04:36:18.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2402076601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3abc026c00 con 0x555df04596c0 2023-06-09T04:36:18.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ab8057cc0 con 0x555df04596c0 2023-06-09T04:36:18.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2402076601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3abc026dc0 con 0x555df04596c0 2023-06-09T04:36:18.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 10 client.5340 fetch_fsmap learned FSMap version 16 2023-06-09T04:36:18.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x555df0201200 con 0x555df04596c0 2023-06-09T04:36:18.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.740+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2402076601 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab8057cc0 con 0x555df04596c0 2023-06-09T04:36:18.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2402076601 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3abc020430 con 0x555df04596c0 2023-06-09T04:36:18.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 10 client.5340 fetch_fsmap finished waiting for FSMap version 16 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 10 client.5340 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 20 client.5340 populate_metadata read hostname 'smithi092' 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df0505fc0 0x555df0506390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 4 client.5340 mds_command: new command op to 5127 tid=0[{"prefix": "status"}] 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 --> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555df0368000 con 0x555df0505fc0 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df0505fc0 0x555df0506390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3aa5ffb700 10 client.5340 ms_handle_connect on v2:172.21.15.92:6835/637518437 2023-06-09T04:36:18.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ac6d9d700 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df0505fc0 0x555df0506390 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb098ad53304e21 server_cookie=f0f1f3a9f5a4366d in_seq=0 out_seq=0 2023-06-09T04:36:18.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3aa5ffb700 1 -- 172.21.15.92:0/2402076601 <== mds.0 v2:172.21.15.92:6835/637518437 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x555df0368000 con 0x555df0505fc0 2023-06-09T04:36:18.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3aa5ffb700 10 client.5340 handle_command_reply: tid=0 2023-06-09T04:36:18.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 client.5340 shutdown 2023-06-09T04:36:18.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 msgr2=0x555df04af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df04af1f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x555df0471090 tx=0x7f3abc004710 comp rx=0 tx=0).stop 2023-06-09T04:36:18.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 shutdown_connections 2023-06-09T04:36:18.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555df04a2c70 0x555df04a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x555df04592f0 0x555df0470280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:6835/637518437,v1:172.21.15.92:6837/637518437] conn(0x555df0505fc0 0x555df0506390 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 --2- 172.21.15.92:0/2402076601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555df04596c0 0x555df04af1f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3aa5ffb700 0 client.5340 ms_handle_reset on v2:172.21.15.92:6835/637518437 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 >> 172.21.15.92:0/2402076601 conn(0x555df0457ce0 msgr2=0x555df04561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 shutdown_connections 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.741+0000 7f3ad933ab80 1 -- 172.21.15.92:0/2402076601 wait complete. 2023-06-09T04:36:18.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3ad933ab80 20 client.5340 trim_cache size 0 max 16384 2023-06-09T04:36:18.753 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-06-09T04:36:18.753 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "064bde79-2943-47e5-b941-05f234bd883f", 2023-06-09T04:36:18.753 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-06-09T04:36:18.753 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5127, 2023-06-09T04:36:18.753 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.9171073099999996, 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 16, 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 38, 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 38, 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 13.32060961 2023-06-09T04:36:18.754 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-06-09T04:36:18.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3ab406ace0 msgr2=0x7f3ab406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3ab406ace0 0x7f3ab406d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3abc006fd0 tx=0x7f3abc008040 comp rx=0 tx=0).stop 2023-06-09T04:36:18.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac80ac400 msgr2=0x7f3ac80c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac80ac400 0x7f3ac80c7bb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8000c00 tx=0x7f3ab8004750 comp rx=0 tx=0).stop 2023-06-09T04:36:18.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 shutdown_connections 2023-06-09T04:36:18.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f3ab406ace0 0x7f3ab406d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3ac815c7d0 0x7f3ac80b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ac80ac400 0x7f3ac80c7bb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 --2- 172.21.15.92:0/1122717382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ac80cf010 0x7f3ac814ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 >> 172.21.15.92:0/1122717382 conn(0x7f3ac8098c10 msgr2=0x7f3ac80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 shutdown_connections 2023-06-09T04:36:18.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.742+0000 7f3acd1b5700 1 -- 172.21.15.92:0/1122717382 wait complete. 2023-06-09T04:36:18.758 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-09T04:36:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.877+0000 7fca0aafe700 1 Processor -- start 2023-06-09T04:36:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.877+0000 7fca0aafe700 1 -- start start 2023-06-09T04:36:18.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.878+0000 7fca0aafe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca0415c850 0x7fca0406ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.878+0000 7fca0aafe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0406ef00 con 0x7fca040cfe60 2023-06-09T04:36:18.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.878+0000 7fca0aafe700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca0406f1a0 con 0x7fca0406b7e0 2023-06-09T04:36:18.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.878+0000 7fca0aafe700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0406f480 con 0x7fca0415c850 2023-06-09T04:36:18.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.878+0000 7fca03fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fca040cfe60 0x7fca0406b0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38080/0 (socket says 172.21.15.92:38080) 2023-06-09T04:36:18.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.878+0000 7fca03fff700 1 -- 172.21.15.92:0/2685364257 learned_addr learned my addr 172.21.15.92:0/2685364257 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:18.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca0889a700 1 --2- 172.21.15.92:0/2685364257 >> v2:172.21.15.123:3300/0 conn(0x7fca0415c850 0x7fca0406ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca037fe700 1 -- 172.21.15.92:0/2685364257 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245673168 0 0) 0x7fca0406ef00 con 0x7fca040cfe60 2023-06-09T04:36:18.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca037fe700 1 -- 172.21.15.92:0/2685364257 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9f40035b0 con 0x7fca040cfe60 2023-06-09T04:36:18.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca037fe700 1 -- 172.21.15.92:0/2685364257 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794560849 0 0) 0x7fc9f40035b0 con 0x7fca040cfe60 2023-06-09T04:36:18.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca0889a700 1 -- 172.21.15.92:0/2685364257 >> v1:172.21.15.123:6789/0 conn(0x7fca0406b7e0 legacy=0x7fca040cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca0889a700 1 -- 172.21.15.92:0/2685364257 >> v1:172.21.15.92:6789/0 conn(0x7fca040cfe60 legacy=0x7fca0406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca0889a700 1 -- 172.21.15.92:0/2685364257 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0406f760 con 0x7fca0415c850 2023-06-09T04:36:18.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca0889a700 1 --2- 172.21.15.92:0/2685364257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0415c850 0x7fca0406ff60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009840 tx=0x7fc9f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d05cc529b3c4ae3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca037fe700 1 -- 172.21.15.92:0/2685364257 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0001670 con 0x7fca0415c850 2023-06-09T04:36:18.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca037fe700 1 -- 172.21.15.92:0/2685364257 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f8004030 con 0x7fca0415c850 2023-06-09T04:36:18.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.879+0000 7fca037fe700 1 -- 172.21.15.92:0/2685364257 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f80043a0 con 0x7fca0415c850 2023-06-09T04:36:18.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.880+0000 7fca0aafe700 1 -- 172.21.15.92:0/2685364257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0415c850 msgr2=0x7fca0406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.880+0000 7fca0aafe700 1 --2- 172.21.15.92:0/2685364257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0415c850 0x7fca0406ff60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009840 tx=0x7fc9f8009b50 comp rx=0 tx=0).stop 2023-06-09T04:36:18.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.880+0000 7fca0aafe700 1 -- 172.21.15.92:0/2685364257 shutdown_connections 2023-06-09T04:36:18.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.880+0000 7fca0aafe700 1 --2- 172.21.15.92:0/2685364257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca0415c850 0x7fca0406ff60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.880+0000 7fca0aafe700 1 -- 172.21.15.92:0/2685364257 >> 172.21.15.92:0/2685364257 conn(0x7fca04089fe0 msgr2=0x7fca040d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.880+0000 7fca0aafe700 1 -- 172.21.15.92:0/2685364257 shutdown_connections 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.880+0000 7fca0aafe700 1 -- 172.21.15.92:0/2685364257 wait complete. 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.881+0000 7fca0aafe700 1 Processor -- start 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.881+0000 7fca0aafe700 1 -- start start 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0aafe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0406b7e0 0x7fca040cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0aafe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca040cfe60 0x7fca040cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0aafe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca0415c850 0x7fca040c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0aafe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0415a960 con 0x7fca0406b7e0 2023-06-09T04:36:18.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0aafe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040cdeb0 con 0x7fca040cfe60 2023-06-09T04:36:18.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0aafe700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca04155930 con 0x7fca0415c850 2023-06-09T04:36:18.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0406b7e0 0x7fca040cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0406b7e0 0x7fca040cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39754/0 (socket says 172.21.15.92:39754) 2023-06-09T04:36:18.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 -- 172.21.15.92:0/3837194213 learned_addr learned my addr 172.21.15.92:0/3837194213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:18.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca03fff700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca040cfe60 0x7fca040cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0909b700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca0415c850 0x7fca040c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 -- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca0415c850 msgr2=0x7fca040c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca0415c850 0x7fca040c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 -- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca040cfe60 msgr2=0x7fca040cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca040cfe60 0x7fca040cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.882+0000 7fca0889a700 1 -- 172.21.15.92:0/3837194213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80094a0 con 0x7fca0406b7e0 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.883+0000 7fca0889a700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0406b7e0 0x7fca040cbb90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8000c00 tx=0x7fc9f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.883+0000 7fca017fa700 1 -- 172.21.15.92:0/3837194213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8003df0 con 0x7fca0406b7e0 2023-06-09T04:36:18.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.883+0000 7fca017fa700 1 -- 172.21.15.92:0/3837194213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f801f070 con 0x7fca0406b7e0 2023-06-09T04:36:18.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.883+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca040cce60 con 0x7fca0406b7e0 2023-06-09T04:36:18.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.883+0000 7fca017fa700 1 -- 172.21.15.92:0/3837194213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8011870 con 0x7fca0406b7e0 2023-06-09T04:36:18.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.883+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca040cc250 con 0x7fca0406b7e0 2023-06-09T04:36:18.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.884+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca040740e0 con 0x7fca0406b7e0 2023-06-09T04:36:18.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.884+0000 7fca017fa700 1 -- 172.21.15.92:0/3837194213 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc9f8021030 con 0x7fca0406b7e0 2023-06-09T04:36:18.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.884+0000 7fca017fa700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc9e806ace0 0x7fc9e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:18.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.884+0000 7fca017fa700 1 -- 172.21.15.92:0/3837194213 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9f808aa40 con 0x7fca0406b7e0 2023-06-09T04:36:18.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.885+0000 7fca03fff700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc9e806ace0 0x7fc9e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:18.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.885+0000 7fca03fff700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc9e806ace0 0x7fc9e806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001eb0 tx=0x7fc9f0001c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:18.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:18.889+0000 7fca017fa700 1 -- 172.21.15.92:0/3837194213 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc9f808ce60 con 0x7fca0406b7e0 2023-06-09T04:36:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.122+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca04073420 con 0x7fca0406b7e0 2023-06-09T04:36:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.122+0000 7fca017fa700 1 -- 172.21.15.92:0/3837194213 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc9f8055c10 con 0x7fca0406b7e0 2023-06-09T04:36:19.124 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-09T04:36:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc9e806ace0 msgr2=0x7fc9e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc9e806ace0 0x7fc9e806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001eb0 tx=0x7fc9f0001c60 comp rx=0 tx=0).stop 2023-06-09T04:36:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0406b7e0 msgr2=0x7fca040cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0406b7e0 0x7fca040cbb90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8000c00 tx=0x7fc9f8004750 comp rx=0 tx=0).stop 2023-06-09T04:36:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 shutdown_connections 2023-06-09T04:36:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fc9e806ace0 0x7fc9e806d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca0415c850 0x7fca040c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca0406b7e0 0x7fca040cbb90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 --2- 172.21.15.92:0/3837194213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca040cfe60 0x7fca040cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 >> 172.21.15.92:0/3837194213 conn(0x7fca04089fe0 msgr2=0x7fca040d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.124+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 shutdown_connections 2023-06-09T04:36:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:19.125+0000 7fca0aafe700 1 -- 172.21.15.92:0/3837194213 wait complete. 2023-06-09T04:36:19.138 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-06-09T04:36:19.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-09T04:36:19.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-06-09T04:36:19.139 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-09T04:36:19.139 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-09T04:36:19.139 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-06-09T04:36:19.139 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-06-09T04:36:19.140 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-09T04:36:19.140 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-09T04:36:19.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.141 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-06-09T04:36:19.157 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T04:36:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T04:36:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:36:19.158 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T04:36:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:36:19.159 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-09T04:36:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:36:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-09T04:36:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42386sec preferred_lft 42386sec 2023-06-09T04:36:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::e3d4:5405:af43:207f/64 scope link noprefixroute 2023-06-09T04:36:19.159 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 2e:c9:69:7c:66:b6 brd ff:ff:ff:ff:ff:ff 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::825:61ff:fe8f:9919/64 scope link 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-09T04:36:19.160 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 2e:c9:69:7c:66:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-09T04:36:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::2cc9:69ff:fe7c:66b6/64 scope link 2023-06-09T04:36:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:36:19.161 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-09T04:36:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 76:38:ac:e6:b8:7e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-09T04:36:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::7438:acff:fee6:b87e/64 scope link 2023-06-09T04:36:19.161 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-06-09T04:36:19.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.162 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-09T04:36:19.222 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-06-09T04:36:19.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.223 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-06-09T04:36:19.280 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-06-09T04:36:19.280 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-09T04:36:19.281 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-09T04:36:19.281 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-09T04:36:19.281 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-09T04:36:19.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.282 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-06-09T04:36:19.282 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-06-09T04:36:19.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-09T04:36:19.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-09T04:36:19.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-09T04:36:19.284 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-06-09T04:36:19.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.386 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-06-09T04:36:19.402 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-06-09T04:36:19.403 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-09T04:36:19.403 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-09T04:36:19.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-06-09T04:36:19.469 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:36:19.470 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-09T04:36:19.525 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-09T04:36:19.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T04:36:19.526 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-09T04:36:19.582 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-06-09T04:36:19.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:36:19.667 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:36:19.667 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:36:19.694 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:36:19.694 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:36:19.694 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:36:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:36:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:36:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:36:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:36:19.695 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:36:19.725 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-06-09T04:36:19.725 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-06-09T04:36:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.123:3300,172.21.15.123:3301" 2023-06-09T04:36:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@064bde79-2943-47e5-b941-05f234bd883f.cephfs=/ 2023-06-09T04:36:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.123:3300/172.21.15.123:3301" will pass to kernel 2023-06-09T04:36:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-06-09T04:36:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-06-09T04:36:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-09T04:36:19.726 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790" 2023-06-09T04:36:19.727 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2023-06-09T04:36:19.727 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=064bde79-2943-47e5-b941-05f234bd883f" will pass to kernel 2023-06-09T04:36:19.744 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-09T04:36:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-09T04:36:19.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-09T04:36:19.855 INFO:teuthology.orchestra.run.smithi092.stdout:client5349 2023-06-09T04:36:19.855 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:36:19.855 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:36:19.913 DEBUG:teuthology.orchestra.run.smithi092:> mktemp -d --suffix=xfstests-dev 2023-06-09T04:36:19.969 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.3HuE1HNmdOxfstests-dev 2023-06-09T04:36:19.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:19.970 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.3HuE1HNmdOxfstests-dev) 2023-06-09T04:36:20.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:20.036 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.3HuE1HNmdOxfstests-dev) 2023-06-09T04:36:20.096 INFO:teuthology.orchestra.run.smithi092.stderr:Cloning into '/tmp/tmp.3HuE1HNmdOxfstests-dev'... 2023-06-09T04:36:20.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:20.757 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-06-09T04:36:20.816 DEBUG:teuthology.orchestra.run.smithi092:> mktemp -d --suffix=test 2023-06-09T04:36:20.873 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.dvoqC6rw3jtest 2023-06-09T04:36:20.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:36:20.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-06-09T04:36:20.932 DEBUG:teuthology.orchestra.run.smithi092:> mktemp -d --suffix=scratch 2023-06-09T04:36:20.987 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:36:20.988 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "rhel" and "8.6". 2023-06-09T04:36:20.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc indent libtool lvm2 make psmisc quota sed xfsdump xfsprogs libacl-devel libattr-devel libaio-devel libuuid-devel xfsprogs-devel python2 sqlite xfsprogs-qa-devel 2023-06-09T04:36:21.385 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-06-09T04:36:22.275 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:04:03 ago on Fri 09 Jun 2023 04:32:19 AM UTC. 2023-06-09T04:36:23.558 INFO:teuthology.orchestra.run.smithi092.stdout:Package acl-2.2.53-1.el8.x86_64 is already installed. 2023-06-09T04:36:23.559 INFO:teuthology.orchestra.run.smithi092.stdout:Package attr-2.4.48-3.el8.x86_64 is already installed. 2023-06-09T04:36:23.560 INFO:teuthology.orchestra.run.smithi092.stdout:Package automake-1.16.1-7.el8.noarch is already installed. 2023-06-09T04:36:23.561 INFO:teuthology.orchestra.run.smithi092.stdout:Package bc-1.07.1-5.el8.x86_64 is already installed. 2023-06-09T04:36:23.562 INFO:teuthology.orchestra.run.smithi092.stdout:Package dbench-4.0-20.el8.x86_64 is already installed. 2023-06-09T04:36:23.564 INFO:teuthology.orchestra.run.smithi092.stdout:Package e2fsprogs-1.45.6-4.el8.x86_64 is already installed. 2023-06-09T04:36:23.566 INFO:teuthology.orchestra.run.smithi092.stdout:Package gawk-4.2.1-4.el8.x86_64 is already installed. 2023-06-09T04:36:23.567 INFO:teuthology.orchestra.run.smithi092.stdout:Package gcc-8.5.0-10.1.el8_6.x86_64 is already installed. 2023-06-09T04:36:23.569 INFO:teuthology.orchestra.run.smithi092.stdout:Package libtool-2.4.6-25.el8.x86_64 is already installed. 2023-06-09T04:36:23.570 INFO:teuthology.orchestra.run.smithi092.stdout:Package lvm2-8:2.03.14-3.el8_6.2.x86_64 is already installed. 2023-06-09T04:36:23.572 INFO:teuthology.orchestra.run.smithi092.stdout:Package make-1:4.2.1-11.el8.x86_64 is already installed. 2023-06-09T04:36:23.573 INFO:teuthology.orchestra.run.smithi092.stdout:Package psmisc-23.1-5.el8.x86_64 is already installed. 2023-06-09T04:36:23.574 INFO:teuthology.orchestra.run.smithi092.stdout:Package quota-1:4.04-14.el8.x86_64 is already installed. 2023-06-09T04:36:23.575 INFO:teuthology.orchestra.run.smithi092.stdout:Package sed-4.5-5.el8.x86_64 is already installed. 2023-06-09T04:36:23.576 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-09T04:36:23.577 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-09T04:36:23.578 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-09T04:36:23.579 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-09T04:36:23.580 INFO:teuthology.orchestra.run.smithi092.stdout:Package libaio-devel-0.3.112-1.el8.x86_64 is already installed. 2023-06-09T04:36:23.581 INFO:teuthology.orchestra.run.smithi092.stdout:Package libuuid-devel-2.32.1-35.el8.x86_64 is already installed. 2023-06-09T04:36:23.582 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-devel-5.0.0-10.el8.x86_64 is already installed. 2023-06-09T04:36:23.585 INFO:teuthology.orchestra.run.smithi092.stdout:Package sqlite-3.26.0-15.el8.x86_64 is already installed. 2023-06-09T04:36:23.589 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-09T04:36:23.704 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-06-09T04:36:23.706 INFO:teuthology.orchestra.run.smithi092.stdout:==================================================================================================================== 2023-06-09T04:36:23.706 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-06-09T04:36:23.706 INFO:teuthology.orchestra.run.smithi092.stdout:==================================================================================================================== 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: dump x86_64 1:0.4-0.36.b46.el8 rhel-8-for-x86_64-baseos-rpms 164 k 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: fio x86_64 3.19-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: indent x86_64 2.2.13-2.el8 epel 187 k 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: python2 x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: sqlite x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 668 k 2023-06-09T04:36:23.707 INFO:teuthology.orchestra.run.smithi092.stdout: sqlite-libs x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 581 k 2023-06-09T04:36:23.708 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-06-09T04:36:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: daxctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 42 k 2023-06-09T04:36:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemblk x86_64 1.6.1-1.el8 rhel-8-for-x86_64-appstream-rpms 92 k 2023-06-09T04:36:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: ndctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 79 k 2023-06-09T04:36:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: python2-libs x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 6.0 M 2023-06-09T04:36:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: python2-pip-wheel noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 892 k 2023-06-09T04:36:23.708 INFO:teuthology.orchestra.run.smithi092.stdout: python2-setuptools-wheel noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 287 k 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: python2-wheel-wheel noarch 1:0.31.1-3.module+el8.5.0+12203+77770ab7 rhel-8-for-x86_64-appstream-rpms 45 k 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: rmt x86_64 2:1.5.3-13.el8 rhel-8-for-x86_64-baseos-rpms 51 k 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: python2-pip noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 1.6 M 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: python2-setuptools noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 642 k 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout:Enabling module streams: 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: python27 2.7 2023-06-09T04:36:23.709 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:23.710 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-06-09T04:36:23.710 INFO:teuthology.orchestra.run.smithi092.stdout:==================================================================================================================== 2023-06-09T04:36:23.710 INFO:teuthology.orchestra.run.smithi092.stdout:Install 14 Packages 2023-06-09T04:36:23.710 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2023-06-09T04:36:23.710 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:23.710 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 12 M 2023-06-09T04:36:23.711 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-06-09T04:36:23.963 INFO:teuthology.orchestra.run.smithi092.stdout:(1/16): python2-setuptools-wheel-39.0.1-13.modu 3.2 MB/s | 287 kB 00:00 2023-06-09T04:36:23.997 INFO:teuthology.orchestra.run.smithi092.stdout:(2/16): python2-setuptools-39.0.1-13.module+el8 5.2 MB/s | 642 kB 00:00 2023-06-09T04:36:24.022 INFO:teuthology.orchestra.run.smithi092.stdout:(3/16): python2-wheel-wheel-0.31.1-3.module+el8 762 kB/s | 45 kB 00:00 2023-06-09T04:36:24.055 INFO:teuthology.orchestra.run.smithi092.stdout:(4/16): fio-3.19-3.el8.x86_64.rpm 9.7 MB/s | 538 kB 00:00 2023-06-09T04:36:24.089 INFO:teuthology.orchestra.run.smithi092.stdout:(5/16): libpmemblk-1.6.1-1.el8.x86_64.rpm 1.3 MB/s | 92 kB 00:00 2023-06-09T04:36:24.117 INFO:teuthology.orchestra.run.smithi092.stdout:(6/16): python2-2.7.18-10.module+el8.6.0+14191+ 1.8 MB/s | 110 kB 00:00 2023-06-09T04:36:24.181 INFO:teuthology.orchestra.run.smithi092.stdout:(7/16): python2-pip-9.0.3-19.module+el8.6.0+130 25 MB/s | 1.6 MB 00:00 2023-06-09T04:36:24.232 INFO:teuthology.orchestra.run.smithi092.stdout:(8/16): python2-pip-wheel-9.0.3-19.module+el8.6 17 MB/s | 892 kB 00:00 2023-06-09T04:36:24.307 INFO:teuthology.orchestra.run.smithi092.stdout:(9/16): python2-libs-2.7.18-10.module+el8.6.0+1 27 MB/s | 6.0 MB 00:00 2023-06-09T04:36:24.332 INFO:teuthology.orchestra.run.smithi092.stdout:(10/16): dump-0.4-0.36.b46.el8.x86_64.rpm 1.6 MB/s | 164 kB 00:00 2023-06-09T04:36:24.349 INFO:teuthology.orchestra.run.smithi092.stdout:(11/16): rmt-1.5.3-13.el8.x86_64.rpm 1.2 MB/s | 51 kB 00:00 2023-06-09T04:36:24.366 INFO:teuthology.orchestra.run.smithi092.stdout:(12/16): daxctl-libs-71.1-3.el8.x86_64.rpm 1.2 MB/s | 42 kB 00:00 2023-06-09T04:36:24.391 INFO:teuthology.orchestra.run.smithi092.stdout:(13/16): ndctl-libs-71.1-3.el8.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2023-06-09T04:36:24.433 INFO:teuthology.orchestra.run.smithi092.stdout:(14/16): sqlite-3.26.0-16.el8_6.x86_64.rpm 9.7 MB/s | 668 kB 00:00 2023-06-09T04:36:24.467 INFO:teuthology.orchestra.run.smithi092.stdout:(15/16): sqlite-libs-3.26.0-16.el8_6.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2023-06-09T04:36:24.684 INFO:teuthology.orchestra.run.smithi092.stdout:(16/16): indent-2.2.13-2.el8.x86_64.rpm 231 kB/s | 187 kB 00:00 2023-06-09T04:36:24.685 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-06-09T04:36:24.685 INFO:teuthology.orchestra.run.smithi092.stdout:Total 12 MB/s | 12 MB 00:00 2023-06-09T04:36:24.891 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-06-09T04:36:24.930 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-06-09T04:36:24.931 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-06-09T04:36:25.115 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-06-09T04:36:25.120 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-06-09T04:36:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-06-09T04:36:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : sqlite-libs-3.26.0-16.el8_6.x86_64 1/18 2023-06-09T04:36:26.179 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : daxctl-libs-71.1-3.el8.x86_64 2/18 2023-06-09T04:36:26.237 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: daxctl-libs-71.1-3.el8.x86_64 2/18 2023-06-09T04:36:26.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ndctl-libs-71.1-3.el8.x86_64 3/18 2023-06-09T04:36:26.503 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ndctl-libs-71.1-3.el8.x86_64 3/18 2023-06-09T04:36:26.704 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-06-09T04:36:26.757 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-06-09T04:36:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rmt-2:1.5.3-13.el8.x86_64 5/18 2023-06-09T04:36:27.100 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 6/18 2023-06-09T04:36:27.359 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 7/18 2023-06-09T04:36:27.979 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 8/18 2023-06-09T04:36:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 9/18 2023-06-09T04:36:28.436 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 10/18 2023-06-09T04:36:29.558 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 10/18 2023-06-09T04:36:29.872 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 11/18 2023-06-09T04:36:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : dump-1:0.4-0.36.b46.el8.x86_64 12/18 2023-06-09T04:36:30.451 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fio-3.19-3.el8.x86_64 13/18 2023-06-09T04:36:30.648 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : sqlite-3.26.0-16.el8_6.x86_64 14/18 2023-06-09T04:36:30.827 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 15/18 2023-06-09T04:36:31.012 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : indent-2.2.13-2.el8.x86_64 16/18 2023-06-09T04:36:31.179 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 17/18 2023-06-09T04:36:31.351 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-06-09T04:36:31.634 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-06-09T04:36:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : indent-2.2.13-2.el8.x86_64 1/18 2023-06-09T04:36:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 2/18 2023-06-09T04:36:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 3/18 2023-06-09T04:36:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 4/18 2023-06-09T04:36:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fio-3.19-3.el8.x86_64 5/18 2023-06-09T04:36:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemblk-1.6.1-1.el8.x86_64 6/18 2023-06-09T04:36:31.635 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 7/18 2023-06-09T04:36:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 8/18 2023-06-09T04:36:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 9/18 2023-06-09T04:36:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 10/18 2023-06-09T04:36:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : dump-1:0.4-0.36.b46.el8.x86_64 11/18 2023-06-09T04:36:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rmt-2:1.5.3-13.el8.x86_64 12/18 2023-06-09T04:36:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : daxctl-libs-71.1-3.el8.x86_64 13/18 2023-06-09T04:36:31.636 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ndctl-libs-71.1-3.el8.x86_64 14/18 2023-06-09T04:36:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : sqlite-3.26.0-16.el8_6.x86_64 15/18 2023-06-09T04:36:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 16/18 2023-06-09T04:36:31.637 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : sqlite-libs-3.26.0-16.el8_6.x86_64 17/18 2023-06-09T04:36:32.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-06-09T04:36:32.959 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:04:13 ago on Fri 09 Jun 2023 04:32:19 AM UTC. 2023-06-09T04:36:35.445 INFO:teuthology.orchestra.run.smithi092.stdout:Installed products updated. 2023-06-09T04:36:35.640 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:35.641 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-06-09T04:36:35.641 INFO:teuthology.orchestra.run.smithi092.stdout: sqlite-3.26.0-16.el8_6.x86_64 sqlite-libs-3.26.0-16.el8_6.x86_64 2023-06-09T04:36:35.641 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-06-09T04:36:35.641 INFO:teuthology.orchestra.run.smithi092.stdout: daxctl-libs-71.1-3.el8.x86_64 2023-06-09T04:36:35.641 INFO:teuthology.orchestra.run.smithi092.stdout: dump-1:0.4-0.36.b46.el8.x86_64 2023-06-09T04:36:35.641 INFO:teuthology.orchestra.run.smithi092.stdout: fio-3.19-3.el8.x86_64 2023-06-09T04:36:35.641 INFO:teuthology.orchestra.run.smithi092.stdout: indent-2.2.13-2.el8.x86_64 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemblk-1.6.1-1.el8.x86_64 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: ndctl-libs-71.1-3.el8.x86_64 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: python2-setuptools-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-06-09T04:36:35.642 INFO:teuthology.orchestra.run.smithi092.stdout: python2-setuptools-wheel-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-06-09T04:36:35.643 INFO:teuthology.orchestra.run.smithi092.stdout: python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+12203+77770ab7.noarch 2023-06-09T04:36:35.643 INFO:teuthology.orchestra.run.smithi092.stdout: rmt-2:1.5.3-13.el8.x86_64 2023-06-09T04:36:35.643 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:35.643 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-06-09T04:36:35.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo useradd -m fsgqa 2023-06-09T04:36:35.815 INFO:teuthology.orchestra.run.smithi092.stderr:useradd: user 'fsgqa' already exists 2023-06-09T04:36:35.816 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-06-09T04:36:35.817 DEBUG:teuthology.orchestra.run.smithi092:> sudo groupadd fsgqa 2023-06-09T04:36:35.884 INFO:teuthology.orchestra.run.smithi092.stderr:groupadd: group 'fsgqa' already exists 2023-06-09T04:36:35.884 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-06-09T04:36:35.885 DEBUG:teuthology.orchestra.run.smithi092:> sudo useradd fsgqa2 2023-06-09T04:36:36.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo useradd 123456-fsgqa 2023-06-09T04:36:37.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-06-09T04:36:37.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.875+0000 7f057ddda700 1 Processor -- start 2023-06-09T04:36:37.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.875+0000 7f057ddda700 1 -- start start 2023-06-09T04:36:37.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.875+0000 7f057ddda700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 0x7f0578070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.875+0000 7f057ddda700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0578069db0 0x7f057806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.875+0000 7f057ddda700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05780c8f30 con 0x7f0578158ac0 2023-06-09T04:36:37.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.875+0000 7f057ddda700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05780cd3b0 con 0x7f0578069db0 2023-06-09T04:36:37.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.875+0000 7f057ddda700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057806cdd0 con 0x7f05781586f0 2023-06-09T04:36:37.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f0576ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0578158ac0 0x7f057806c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38116/0 (socket says 172.21.15.92:38116) 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f0576ffd700 1 -- 172.21.15.92:0/3830706167 learned_addr learned my addr 172.21.15.92:0/3830706167 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320286365 0 0) 0x7f05780c8f30 con 0x7f0578158ac0 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05600035b0 con 0x7f0578158ac0 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f0577fff700 1 --2- 172.21.15.92:0/3830706167 >> v2:172.21.15.123:3300/0 conn(0x7f0578069db0 0x7f057806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05777fe700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 0x7f0578070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 326358043 0 0) 0x7f05600035b0 con 0x7f0578158ac0 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 msgr2=0x7f0578070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:37.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 0x7f0578070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:37.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 >> v2:172.21.15.123:3300/0 conn(0x7f0578069db0 msgr2=0x7f057806a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:37.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 --2- 172.21.15.92:0/3830706167 >> v2:172.21.15.123:3300/0 conn(0x7f0578069db0 0x7f057806a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:37.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05780c97b0 con 0x7f0578158ac0 2023-06-09T04:36:37.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7f056c001670 con 0x7f0578158ac0 2023-06-09T04:36:37.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 >> v1:172.21.15.92:6789/0 conn(0x7f0578158ac0 legacy=0x7f057806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:37.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 0x7f0560003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560003f80 0x7f0560004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0560004d90 0x7f0560005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05780cd3b0 con 0x7f0560004d90 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057806cdd0 con 0x7f0560003f80 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056000ba40 con 0x7f05781586f0 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f0577fff700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0560004d90 0x7f0560005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f0576ffd700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560003f80 0x7f0560004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.876+0000 7f05777fe700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 0x7f0560003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f0577fff700 1 -- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 msgr2=0x7f0560003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f0577fff700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05781586f0 0x7f0560003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f0577fff700 1 -- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560003f80 msgr2=0x7f0560004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f0577fff700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560003f80 0x7f0560004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f0577fff700 1 -- 172.21.15.92:0/3830706167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600035b0 con 0x7f0560004d90 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f0577fff700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0560004d90 0x7f0560005200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0570009fc0 tx=0x7f0560005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14c4d102b812fdc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057000ec30 con 0x7f0560004d90 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057000f040 con 0x7f0560004d90 2023-06-09T04:36:37.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f05767fc700 1 -- 172.21.15.92:0/3830706167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570014480 con 0x7f0560004d90 2023-06-09T04:36:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f057ddda700 1 -- 172.21.15.92:0/3830706167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0560004d90 msgr2=0x7f0560005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f057ddda700 1 --2- 172.21.15.92:0/3830706167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0560004d90 0x7f0560005200 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0570009fc0 tx=0x7f0560005880 comp rx=0 tx=0).stop 2023-06-09T04:36:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f0577fff700 1 -- 172.21.15.92:0/3830706167 reap_dead start 2023-06-09T04:36:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f057ddda700 1 -- 172.21.15.92:0/3830706167 shutdown_connections 2023-06-09T04:36:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f057ddda700 1 -- 172.21.15.92:0/3830706167 >> 172.21.15.92:0/3830706167 conn(0x7f05780b0ff0 msgr2=0x7f05780ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f057ddda700 1 -- 172.21.15.92:0/3830706167 shutdown_connections 2023-06-09T04:36:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.877+0000 7f057ddda700 1 -- 172.21.15.92:0/3830706167 wait complete. 2023-06-09T04:36:37.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.879+0000 7f057ddda700 1 Processor -- start 2023-06-09T04:36:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.879+0000 7f057ddda700 1 -- start start 2023-06-09T04:36:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560004d90 0x7f0578156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05780b34e0 0x7f05780b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05780cb5c0 0x7f05780cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05780cbf50 con 0x7f05780cb5c0 2023-06-09T04:36:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05780b1f20 con 0x7f0560004d90 2023-06-09T04:36:37.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05780b2200 con 0x7f05780b34e0 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05780cb5c0 0x7f05780cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05780cb5c0 0x7f05780cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39800/0 (socket says 172.21.15.92:39800) 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 -- 172.21.15.92:0/512984920 learned_addr learned my addr 172.21.15.92:0/512984920 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f05777fe700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560004d90 0x7f0578156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0576ffd700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05780b34e0 0x7f05780b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 -- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05780b34e0 msgr2=0x7f05780b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05780b34e0 0x7f05780b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:37.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 -- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560004d90 msgr2=0x7f0578156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:37.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560004d90 0x7f0578156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:37.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 -- 172.21.15.92:0/512984920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05780b24e0 con 0x7f05780cb5c0 2023-06-09T04:36:37.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f0577fff700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05780cb5c0 0x7f05780cba10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0570017040 tx=0x7f0570009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:37.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f055ffff700 1 -- 172.21.15.92:0/512984920 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570004450 con 0x7f05780cb5c0 2023-06-09T04:36:37.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f055ffff700 1 -- 172.21.15.92:0/512984920 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05700045f0 con 0x7f05780cb5c0 2023-06-09T04:36:37.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05780b27c0 con 0x7f05780cb5c0 2023-06-09T04:36:37.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f055ffff700 1 -- 172.21.15.92:0/512984920 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570003e10 con 0x7f05780cb5c0 2023-06-09T04:36:37.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.880+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0578157420 con 0x7f05780cb5c0 2023-06-09T04:36:37.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.881+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057806e1c0 con 0x7f05780cb5c0 2023-06-09T04:36:37.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.882+0000 7f055ffff700 1 -- 172.21.15.92:0/512984920 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0570024070 con 0x7f05780cb5c0 2023-06-09T04:36:37.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.882+0000 7f055ffff700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f056406acd0 0x7f056406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:37.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.882+0000 7f055ffff700 1 -- 172.21.15.92:0/512984920 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f057008af40 con 0x7f05780cb5c0 2023-06-09T04:36:37.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.882+0000 7f05777fe700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f056406acd0 0x7f056406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:37.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.882+0000 7f05777fe700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f056406acd0 0x7f056406d180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0568009ba0 tx=0x7f0568008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:37.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:37.887+0000 7f055ffff700 1 -- 172.21.15.92:0/512984920 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0570056080 con 0x7f05780cb5c0 2023-06-09T04:36:38.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.155+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0578066d50 con 0x7f05780cb5c0 2023-06-09T04:36:38.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.156+0000 7f055ffff700 1 -- 172.21.15.92:0/512984920 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f0570018510 con 0x7f05780cb5c0 2023-06-09T04:36:38.156 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:36:38.156 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":1,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","modified":"2023-06-09T04:34:17.728261Z","created":"2023-06-09T04:34:17.728261Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3301","nonce":0},{"type":"v1","addr":"172.21.15.123:6790","nonce":0}]},"addr":"172.21.15.123:6790/0","public_addr":"172.21.15.123:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-06-09T04:36:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f056406acd0 msgr2=0x7f056406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f056406acd0 0x7f056406d180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0568009ba0 tx=0x7f0568008040 comp rx=0 tx=0).stop 2023-06-09T04:36:38.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05780cb5c0 msgr2=0x7f05780cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05780cb5c0 0x7f05780cba10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0570017040 tx=0x7f0570009750 comp rx=0 tx=0).stop 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 shutdown_connections 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f056406acd0 0x7f056406d180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05780cb5c0 0x7f05780cba10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0560004d90 0x7f0578156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 --2- 172.21.15.92:0/512984920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05780b34e0 0x7f05780b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 >> 172.21.15.92:0/512984920 conn(0x7f05780b0ff0 msgr2=0x7f05780ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:38.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 shutdown_connections 2023-06-09T04:36:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.158+0000 7f057ddda700 1 -- 172.21.15.92:0/512984920 wait complete. 2023-06-09T04:36:38.160 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 1 2023-06-09T04:36:38.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2023-06-09T04:36:38.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.311+0000 7ff500505700 1 Processor -- start 2023-06-09T04:36:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.311+0000 7ff500505700 1 -- start start 2023-06-09T04:36:38.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff500505700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4f80ae630 0x7ff4f80aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:38.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff500505700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4f80cd7e0 0x7ff4f80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:38.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff500505700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80adde0 con 0x7ff4f80b1360 2023-06-09T04:36:38.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff500505700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80ae270 con 0x7ff4f80ae630 2023-06-09T04:36:38.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff500505700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80af0f0 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff4fdaa0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4f80ae630 0x7ff4f80aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:38.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff4fdaa0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4f80ae630 0x7ff4f80aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39806/0 (socket says 172.21.15.92:39806) 2023-06-09T04:36:38.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff4fdaa0700 1 -- 172.21.15.92:0/891274236 learned_addr learned my addr 172.21.15.92:0/891274236 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:38.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff4feaa2700 1 --2- 172.21.15.92:0/891274236 >> v2:172.21.15.123:3300/0 conn(0x7ff4f80cd7e0 0x7ff4f80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:38.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff4fdaa0700 1 -- 172.21.15.92:0/891274236 >> v2:172.21.15.123:3300/0 conn(0x7ff4f80cd7e0 msgr2=0x7ff4f80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff4fdaa0700 1 --2- 172.21.15.92:0/891274236 >> v2:172.21.15.123:3300/0 conn(0x7ff4f80cd7e0 0x7ff4f80cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.312+0000 7ff4fdaa0700 1 -- 172.21.15.92:0/891274236 >> v1:172.21.15.123:6789/0 conn(0x7ff4f80b1360 legacy=0x7ff4f80b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:36:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.313+0000 7ff4fdaa0700 1 -- 172.21.15.92:0/891274236 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80ad4b0 con 0x7ff4f80ae630 2023-06-09T04:36:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.313+0000 7ff4fdaa0700 1 --2- 172.21.15.92:0/891274236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80ae630 0x7ff4f80aeaa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8071660 tx=0x7ff4e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191b0d40199dc8b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.313+0000 7ff4fd29f700 1 -- 172.21.15.92:0/891274236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004230 con 0x7ff4f80ae630 2023-06-09T04:36:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.313+0000 7ff4fd29f700 1 -- 172.21.15.92:0/891274236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e8004390 con 0x7ff4f80ae630 2023-06-09T04:36:38.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.313+0000 7ff4fd29f700 1 -- 172.21.15.92:0/891274236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80036a0 con 0x7ff4f80ae630 2023-06-09T04:36:38.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.313+0000 7ff500505700 1 -- 172.21.15.92:0/891274236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80ae630 msgr2=0x7ff4f80aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:38.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.313+0000 7ff500505700 1 --2- 172.21.15.92:0/891274236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80ae630 0x7ff4f80aeaa0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8071660 tx=0x7ff4e8009bd0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.314+0000 7ff500505700 1 -- 172.21.15.92:0/891274236 shutdown_connections 2023-06-09T04:36:38.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.314+0000 7ff500505700 1 --2- 172.21.15.92:0/891274236 >> v2:172.21.15.123:3300/0 conn(0x7ff4f80cd7e0 0x7ff4f80cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.314+0000 7ff500505700 1 --2- 172.21.15.92:0/891274236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80ae630 0x7ff4f80aeaa0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.314+0000 7ff500505700 1 -- 172.21.15.92:0/891274236 >> 172.21.15.92:0/891274236 conn(0x7ff4f8098c10 msgr2=0x7ff4f80cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:38.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.314+0000 7ff500505700 1 -- 172.21.15.92:0/891274236 shutdown_connections 2023-06-09T04:36:38.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.314+0000 7ff500505700 1 -- 172.21.15.92:0/891274236 wait complete. 2023-06-09T04:36:38.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.315+0000 7ff500505700 1 Processor -- start 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.315+0000 7ff500505700 1 -- start start 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.315+0000 7ff500505700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4f80ae630 0x7ff4f80c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.315+0000 7ff500505700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f80b1360 0x7ff4f81ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff500505700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80cd7e0 0x7ff4f81ef860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff500505700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f8149af0 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff500505700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80c7ee0 con 0x7ff4f80b1360 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff500505700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80cc820 con 0x7ff4f80ae630 2023-06-09T04:36:38.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80cd7e0 0x7ff4f81ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80cd7e0 0x7ff4f81ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39814/0 (socket says 172.21.15.92:39814) 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 -- 172.21.15.92:0/3748865307 learned_addr learned my addr 172.21.15.92:0/3748865307 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4fdaa0700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f80b1360 0x7ff4f81ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4fe2a1700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4f80ae630 0x7ff4f80c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 -- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4f80ae630 msgr2=0x7ff4f80c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4f80ae630 0x7ff4f80c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 -- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f80b1360 msgr2=0x7ff4f81ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:38.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f80b1360 0x7ff4f81ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 -- 172.21.15.92:0/3748865307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e8009550 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4feaa2700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80cd7e0 0x7ff4f81ef860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000cc30 tx=0x7ff4f000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4eeffd700 1 -- 172.21.15.92:0/3748865307 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f000aa50 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4eeffd700 1 -- 172.21.15.92:0/3748865307 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f000abb0 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff4eeffd700 1 -- 172.21.15.92:0/3748865307 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0003a20 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f80ccb00 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.316+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f80ccde0 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.317+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f80c7d40 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.318+0000 7ff4eeffd700 1 -- 172.21.15.92:0/3748865307 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff4f000f3f0 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.318+0000 7ff4eeffd700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff4dc06ace0 0x7ff4dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:36:38.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.318+0000 7ff4eeffd700 1 -- 172.21.15.92:0/3748865307 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4f0089530 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.318+0000 7ff4fe2a1700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff4dc06ace0 0x7ff4dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:36:38.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.318+0000 7ff4fe2a1700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff4dc06ace0 0x7ff4dc06d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40050c0 tx=0x7ff4f4007f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:36:38.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.322+0000 7ff4eeffd700 1 -- 172.21.15.92:0/3748865307 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff4f80c7d40 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.595+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7ff4f80c83c0 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.596+0000 7ff4eeffd700 1 -- 172.21.15.92:0/3748865307 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v15) v1 ==== 92+0+63 (secure 0 0 0) 0x7ff4f80c83c0 con 0x7ff4f80cd7e0 2023-06-09T04:36:38.597 INFO:teuthology.orchestra.run.smithi092.stdout:[client.admin] 2023-06-09T04:36:38.597 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== 2023-06-09T04:36:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff4dc06ace0 msgr2=0x7ff4dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:38.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff4dc06ace0 0x7ff4dc06d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40050c0 tx=0x7ff4f4007f10 comp rx=0 tx=0).stop 2023-06-09T04:36:38.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80cd7e0 msgr2=0x7ff4f81ef860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:36:38.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80cd7e0 0x7ff4f81ef860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000cc30 tx=0x7ff4f000cf40 comp rx=0 tx=0).stop 2023-06-09T04:36:38.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 shutdown_connections 2023-06-09T04:36:38.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7ff4dc06ace0 0x7ff4dc06d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f80cd7e0 0x7ff4f81ef860 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4f80ae630 0x7ff4f80c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 --2- 172.21.15.92:0/3748865307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f80b1360 0x7ff4f81ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:36:38.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 >> 172.21.15.92:0/3748865307 conn(0x7ff4f8098c10 msgr2=0x7ff4f806d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:36:38.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 shutdown_connections 2023-06-09T04:36:38.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:36:38.598+0000 7ff500505700 1 -- 172.21.15.92:0/3748865307 wait complete. 2023-06-09T04:36:38.611 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:36:38.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/tmp/tmp.3HuE1HNmdOxfstests-dev/local.config 2023-06-09T04:36:38.649 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.92:6789:/test export TEST_DIR=/tmp/tmp.dvoqC6rw3jtest export SCRATCH_DEV=172.21.15.92:6789:/scratch export SCRATCH_MNT=/tmp/tmp.avGnPmg37Fscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA==" 2023-06-09T04:36:38.650 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:36:38.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/tmp/tmp.3HuE1HNmdOxfstests-dev/ceph.exclude 2023-06-09T04:36:38.715 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo make) 2023-06-09T04:36:38.787 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize -cfi 2023-06-09T04:36:39.109 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: putting auxiliary files in '.'. 2023-06-09T04:36:39.109 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file './config.guess' 2023-06-09T04:36:39.112 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file './config.sub' 2023-06-09T04:36:39.116 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file './install-sh' 2023-06-09T04:36:39.119 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file './ltmain.sh' 2023-06-09T04:36:39.139 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-06-09T04:36:39.139 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-06-09T04:36:39.143 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-06-09T04:36:39.148 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-06-09T04:36:39.152 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-06-09T04:36:39.156 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-06-09T04:36:39.162 INFO:teuthology.orchestra.run.smithi092.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-06-09T04:36:39.163 INFO:teuthology.orchestra.run.smithi092.stdout:cp include/install-sh . 2023-06-09T04:36:39.164 INFO:teuthology.orchestra.run.smithi092.stdout:aclocal -I m4 2023-06-09T04:36:40.304 INFO:teuthology.orchestra.run.smithi092.stdout:autoheader 2023-06-09T04:36:40.598 INFO:teuthology.orchestra.run.smithi092.stdout:autoconf 2023-06-09T04:36:40.756 INFO:teuthology.orchestra.run.smithi092.stdout:./configure \ 2023-06-09T04:36:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: --libexecdir=/usr/lib \ 2023-06-09T04:36:40.756 INFO:teuthology.orchestra.run.smithi092.stdout: --exec_prefix=/var/lib 2023-06-09T04:36:41.190 INFO:teuthology.orchestra.run.smithi092.stdout:checking build system type... x86_64-pc-linux-gnu 2023-06-09T04:36:41.191 INFO:teuthology.orchestra.run.smithi092.stdout:checking host system type... x86_64-pc-linux-gnu 2023-06-09T04:36:41.192 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to print strings... printf 2023-06-09T04:36:41.193 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gcc... gcc 2023-06-09T04:36:41.308 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether the C compiler works... yes 2023-06-09T04:36:41.309 INFO:teuthology.orchestra.run.smithi092.stdout:checking for C compiler default output file name... a.out 2023-06-09T04:36:41.336 INFO:teuthology.orchestra.run.smithi092.stdout:checking for suffix of executables... 2023-06-09T04:36:41.371 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether we are cross compiling... no 2023-06-09T04:36:41.389 INFO:teuthology.orchestra.run.smithi092.stdout:checking for suffix of object files... o 2023-06-09T04:36:41.405 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether we are using the GNU C compiler... yes 2023-06-09T04:36:41.421 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether gcc accepts -g... yes 2023-06-09T04:36:41.447 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gcc option to accept ISO C89... none needed 2023-06-09T04:36:41.450 INFO:teuthology.orchestra.run.smithi092.stdout:checking for a sed that does not truncate output... /bin/sed 2023-06-09T04:36:41.452 INFO:teuthology.orchestra.run.smithi092.stdout:checking for grep that handles long lines and -e... /bin/grep 2023-06-09T04:36:41.453 INFO:teuthology.orchestra.run.smithi092.stdout:checking for egrep... /bin/grep -E 2023-06-09T04:36:41.454 INFO:teuthology.orchestra.run.smithi092.stdout:checking for fgrep... /bin/grep -F 2023-06-09T04:36:41.457 INFO:teuthology.orchestra.run.smithi092.stdout:checking for ld used by gcc... /bin/ld 2023-06-09T04:36:41.459 INFO:teuthology.orchestra.run.smithi092.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2023-06-09T04:36:41.468 INFO:teuthology.orchestra.run.smithi092.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2023-06-09T04:36:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2023-06-09T04:36:41.484 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether ln -s works... yes 2023-06-09T04:36:41.490 INFO:teuthology.orchestra.run.smithi092.stdout:checking the maximum length of command line arguments... 1572864 2023-06-09T04:36:41.492 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-06-09T04:36:41.492 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-06-09T04:36:41.492 INFO:teuthology.orchestra.run.smithi092.stdout:checking for /bin/ld option to reload object files... -r 2023-06-09T04:36:41.493 INFO:teuthology.orchestra.run.smithi092.stdout:checking for objdump... objdump 2023-06-09T04:36:41.493 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to recognize dependent libraries... pass_all 2023-06-09T04:36:41.493 INFO:teuthology.orchestra.run.smithi092.stdout:checking for dlltool... no 2023-06-09T04:36:41.494 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-06-09T04:36:41.494 INFO:teuthology.orchestra.run.smithi092.stdout:checking for ar... ar 2023-06-09T04:36:41.520 INFO:teuthology.orchestra.run.smithi092.stdout:checking for archiver @FILE support... @ 2023-06-09T04:36:41.521 INFO:teuthology.orchestra.run.smithi092.stdout:checking for strip... strip 2023-06-09T04:36:41.521 INFO:teuthology.orchestra.run.smithi092.stdout:checking for ranlib... ranlib 2023-06-09T04:36:41.521 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gawk... gawk 2023-06-09T04:36:41.571 INFO:teuthology.orchestra.run.smithi092.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2023-06-09T04:36:41.572 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sysroot... no 2023-06-09T04:36:41.575 INFO:teuthology.orchestra.run.smithi092.stdout:checking for a working dd... /bin/dd 2023-06-09T04:36:41.577 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2023-06-09T04:36:41.590 INFO:teuthology.orchestra.run.smithi092.stdout:checking for mt... no 2023-06-09T04:36:41.592 INFO:teuthology.orchestra.run.smithi092.stdout:checking if : is a manifest tool... no 2023-06-09T04:36:41.629 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to run the C preprocessor... gcc -E 2023-06-09T04:36:41.751 INFO:teuthology.orchestra.run.smithi092.stdout:checking for ANSI C header files... yes 2023-06-09T04:36:41.770 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/types.h... yes 2023-06-09T04:36:41.793 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/stat.h... yes 2023-06-09T04:36:41.815 INFO:teuthology.orchestra.run.smithi092.stdout:checking for stdlib.h... yes 2023-06-09T04:36:41.839 INFO:teuthology.orchestra.run.smithi092.stdout:checking for string.h... yes 2023-06-09T04:36:41.863 INFO:teuthology.orchestra.run.smithi092.stdout:checking for memory.h... yes 2023-06-09T04:36:41.887 INFO:teuthology.orchestra.run.smithi092.stdout:checking for strings.h... yes 2023-06-09T04:36:41.912 INFO:teuthology.orchestra.run.smithi092.stdout:checking for inttypes.h... yes 2023-06-09T04:36:41.936 INFO:teuthology.orchestra.run.smithi092.stdout:checking for stdint.h... yes 2023-06-09T04:36:41.964 INFO:teuthology.orchestra.run.smithi092.stdout:checking for unistd.h... yes 2023-06-09T04:36:41.989 INFO:teuthology.orchestra.run.smithi092.stdout:checking for dlfcn.h... yes 2023-06-09T04:36:41.992 INFO:teuthology.orchestra.run.smithi092.stdout:checking for objdir... .libs 2023-06-09T04:36:42.056 INFO:teuthology.orchestra.run.smithi092.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-06-09T04:36:42.057 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-06-09T04:36:42.071 INFO:teuthology.orchestra.run.smithi092.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-06-09T04:36:42.091 INFO:teuthology.orchestra.run.smithi092.stdout:checking if gcc static flag -static works... no 2023-06-09T04:36:42.111 INFO:teuthology.orchestra.run.smithi092.stdout:checking if gcc supports -c -o file.o... yes 2023-06-09T04:36:42.111 INFO:teuthology.orchestra.run.smithi092.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-06-09T04:36:42.117 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-06-09T04:36:42.141 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether -lc should be explicitly linked in... no 2023-06-09T04:36:42.185 INFO:teuthology.orchestra.run.smithi092.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-06-09T04:36:42.185 INFO:teuthology.orchestra.run.smithi092.stdout:checking how to hardcode library paths into programs... immediate 2023-06-09T04:36:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether stripping libraries is possible... yes 2023-06-09T04:36:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:checking if libtool supports shared libraries... yes 2023-06-09T04:36:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether to build shared libraries... yes 2023-06-09T04:36:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether to build static libraries... yes 2023-06-09T04:36:42.191 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gcc... (cached) gcc 2023-06-09T04:36:42.206 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-06-09T04:36:42.207 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether gcc accepts -g... (cached) yes 2023-06-09T04:36:42.207 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-06-09T04:36:42.207 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gmake... /usr/bin/gmake 2023-06-09T04:36:42.207 INFO:teuthology.orchestra.run.smithi092.stderr:./configure: line 12070: test: $(SHELL): binary operator expected 2023-06-09T04:36:42.208 INFO:teuthology.orchestra.run.smithi092.stdout:checking for tar... /bin/tar 2023-06-09T04:36:42.208 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gzip... /bin/gzip 2023-06-09T04:36:42.214 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether gcc -MM is supported... yes 2023-06-09T04:36:42.215 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sort... /bin/sort 2023-06-09T04:36:42.215 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether ln -s works... yes 2023-06-09T04:36:42.215 INFO:teuthology.orchestra.run.smithi092.stdout:checking for rpm... /bin/rpm 2023-06-09T04:36:42.219 INFO:teuthology.orchestra.run.smithi092.stdout:checking for rpmbuild... no 2023-06-09T04:36:42.243 INFO:teuthology.orchestra.run.smithi092.stdout:checking assert.h usability... yes 2023-06-09T04:36:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:checking assert.h presence... yes 2023-06-09T04:36:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:checking for assert.h... yes 2023-06-09T04:36:42.275 INFO:teuthology.orchestra.run.smithi092.stdout:checking bstring.h usability... no 2023-06-09T04:36:42.284 INFO:teuthology.orchestra.run.smithi092.stdout:checking bstring.h presence... no 2023-06-09T04:36:42.284 INFO:teuthology.orchestra.run.smithi092.stdout:checking for bstring.h... no 2023-06-09T04:36:42.308 INFO:teuthology.orchestra.run.smithi092.stdout:checking libgen.h usability... yes 2023-06-09T04:36:42.314 INFO:teuthology.orchestra.run.smithi092.stdout:checking libgen.h presence... yes 2023-06-09T04:36:42.314 INFO:teuthology.orchestra.run.smithi092.stdout:checking for libgen.h... yes 2023-06-09T04:36:42.341 INFO:teuthology.orchestra.run.smithi092.stdout:checking dirent.h usability... yes 2023-06-09T04:36:42.349 INFO:teuthology.orchestra.run.smithi092.stdout:checking dirent.h presence... yes 2023-06-09T04:36:42.349 INFO:teuthology.orchestra.run.smithi092.stdout:checking for dirent.h... yes 2023-06-09T04:36:42.407 INFO:teuthology.orchestra.run.smithi092.stdout:checking errno.h usability... yes 2023-06-09T04:36:42.414 INFO:teuthology.orchestra.run.smithi092.stdout:checking errno.h presence... yes 2023-06-09T04:36:42.414 INFO:teuthology.orchestra.run.smithi092.stdout:checking for errno.h... yes 2023-06-09T04:36:42.440 INFO:teuthology.orchestra.run.smithi092.stdout:checking malloc.h usability... yes 2023-06-09T04:36:42.448 INFO:teuthology.orchestra.run.smithi092.stdout:checking malloc.h presence... yes 2023-06-09T04:36:42.449 INFO:teuthology.orchestra.run.smithi092.stdout:checking for malloc.h... yes 2023-06-09T04:36:42.474 INFO:teuthology.orchestra.run.smithi092.stdout:checking uuid.h usability... no 2023-06-09T04:36:42.483 INFO:teuthology.orchestra.run.smithi092.stdout:checking uuid.h presence... no 2023-06-09T04:36:42.483 INFO:teuthology.orchestra.run.smithi092.stdout:checking for uuid.h... no 2023-06-09T04:36:42.509 INFO:teuthology.orchestra.run.smithi092.stdout:checking uuid/uuid.h usability... yes 2023-06-09T04:36:42.517 INFO:teuthology.orchestra.run.smithi092.stdout:checking uuid/uuid.h presence... yes 2023-06-09T04:36:42.517 INFO:teuthology.orchestra.run.smithi092.stdout:checking for uuid/uuid.h... yes 2023-06-09T04:36:42.542 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/uuid.h usability... no 2023-06-09T04:36:42.552 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/uuid.h presence... no 2023-06-09T04:36:42.552 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/uuid.h... no 2023-06-09T04:36:42.576 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/file.h usability... yes 2023-06-09T04:36:42.584 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/file.h presence... yes 2023-06-09T04:36:42.584 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/file.h... yes 2023-06-09T04:36:42.610 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/fcntl.h usability... yes 2023-06-09T04:36:42.618 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/fcntl.h presence... yes 2023-06-09T04:36:42.618 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/fcntl.h... yes 2023-06-09T04:36:42.643 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/syssgi.h usability... no 2023-06-09T04:36:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/syssgi.h presence... no 2023-06-09T04:36:42.653 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/syssgi.h... no 2023-06-09T04:36:42.691 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/param.h usability... yes 2023-06-09T04:36:42.701 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/param.h presence... yes 2023-06-09T04:36:42.701 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/param.h... yes 2023-06-09T04:36:42.706 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/stat.h... (cached) yes 2023-06-09T04:36:42.732 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/statvfs.h usability... yes 2023-06-09T04:36:42.740 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/statvfs.h presence... yes 2023-06-09T04:36:42.740 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/statvfs.h... yes 2023-06-09T04:36:42.766 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/time.h usability... yes 2023-06-09T04:36:42.773 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/time.h presence... yes 2023-06-09T04:36:42.773 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/time.h... yes 2023-06-09T04:36:42.802 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/ioctl.h usability... yes 2023-06-09T04:36:42.809 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/ioctl.h presence... yes 2023-06-09T04:36:42.810 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/ioctl.h... yes 2023-06-09T04:36:42.838 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/wait.h usability... yes 2023-06-09T04:36:42.846 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/wait.h presence... yes 2023-06-09T04:36:42.846 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/wait.h... yes 2023-06-09T04:36:42.851 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/types.h... (cached) yes 2023-06-09T04:36:42.856 INFO:teuthology.orchestra.run.smithi092.stdout:checking for strings.h... (cached) yes 2023-06-09T04:36:42.890 INFO:teuthology.orchestra.run.smithi092.stdout:checking err.h usability... yes 2023-06-09T04:36:42.897 INFO:teuthology.orchestra.run.smithi092.stdout:checking err.h presence... yes 2023-06-09T04:36:42.897 INFO:teuthology.orchestra.run.smithi092.stdout:checking for err.h... yes 2023-06-09T04:36:42.923 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/falloc.h usability... yes 2023-06-09T04:36:42.929 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/falloc.h presence... yes 2023-06-09T04:36:42.929 INFO:teuthology.orchestra.run.smithi092.stdout:checking for linux/falloc.h... yes 2023-06-09T04:36:42.955 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-06-09T04:36:42.964 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-06-09T04:36:42.964 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/fs/xfs_fsops.h... no 2023-06-09T04:36:42.986 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/fs/xfs_itable.h usability... no 2023-06-09T04:36:42.995 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/fs/xfs_itable.h presence... no 2023-06-09T04:36:42.995 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/fs/xfs_itable.h... no 2023-06-09T04:36:43.018 INFO:teuthology.orchestra.run.smithi092.stdout:checking xfs/platform_defs.h usability... no 2023-06-09T04:36:43.027 INFO:teuthology.orchestra.run.smithi092.stdout:checking xfs/platform_defs.h presence... no 2023-06-09T04:36:43.028 INFO:teuthology.orchestra.run.smithi092.stdout:checking for xfs/platform_defs.h... no 2023-06-09T04:36:43.050 INFO:teuthology.orchestra.run.smithi092.stdout:checking btrfs/ioctl.h usability... no 2023-06-09T04:36:43.059 INFO:teuthology.orchestra.run.smithi092.stdout:checking btrfs/ioctl.h presence... no 2023-06-09T04:36:43.059 INFO:teuthology.orchestra.run.smithi092.stdout:checking for btrfs/ioctl.h... no 2023-06-09T04:36:43.081 INFO:teuthology.orchestra.run.smithi092.stdout:checking cifs/ioctl.h usability... no 2023-06-09T04:36:43.090 INFO:teuthology.orchestra.run.smithi092.stdout:checking cifs/ioctl.h presence... no 2023-06-09T04:36:43.090 INFO:teuthology.orchestra.run.smithi092.stdout:checking for cifs/ioctl.h... no 2023-06-09T04:36:43.131 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/mman.h usability... yes 2023-06-09T04:36:43.138 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/mman.h presence... yes 2023-06-09T04:36:43.138 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/mman.h... yes 2023-06-09T04:36:43.153 INFO:teuthology.orchestra.run.smithi092.stdout:checking for xfs/xfs_log_format.h... no 2023-06-09T04:36:43.186 INFO:teuthology.orchestra.run.smithi092.stdout:checking for uuid_compare... no 2023-06-09T04:36:43.216 INFO:teuthology.orchestra.run.smithi092.stdout:checking for uuid_compare in -luuid... yes 2023-06-09T04:36:43.261 INFO:teuthology.orchestra.run.smithi092.stdout:checking for xfs/xfs.h... yes 2023-06-09T04:36:43.274 INFO:teuthology.orchestra.run.smithi092.stdout:checking for xfs/libxfs.h... no 2023-06-09T04:36:43.288 INFO:teuthology.orchestra.run.smithi092.stdout:checking whether xlog_assign_lsn is declared... no 2023-06-09T04:36:43.314 INFO:teuthology.orchestra.run.smithi092.stdout:checking for xfs/xqm.h... yes 2023-06-09T04:36:43.358 INFO:teuthology.orchestra.run.smithi092.stdout:checking xfsctl from xfs/xfs.h... ok 2023-06-09T04:36:43.380 INFO:teuthology.orchestra.run.smithi092.stdout:checking xfs/handle.h usability... yes 2023-06-09T04:36:43.387 INFO:teuthology.orchestra.run.smithi092.stdout:checking xfs/handle.h presence... yes 2023-06-09T04:36:43.387 INFO:teuthology.orchestra.run.smithi092.stdout:checking for xfs/handle.h... yes 2023-06-09T04:36:43.423 INFO:teuthology.orchestra.run.smithi092.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-06-09T04:36:43.456 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/acl.h usability... yes 2023-06-09T04:36:43.464 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/acl.h presence... yes 2023-06-09T04:36:43.464 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/acl.h... yes 2023-06-09T04:36:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:checking acl/libacl.h usability... yes 2023-06-09T04:36:43.506 INFO:teuthology.orchestra.run.smithi092.stdout:checking acl/libacl.h presence... yes 2023-06-09T04:36:43.506 INFO:teuthology.orchestra.run.smithi092.stdout:checking for acl/libacl.h... yes 2023-06-09T04:36:43.537 INFO:teuthology.orchestra.run.smithi092.stdout:checking for acl_init in -lacl... yes 2023-06-09T04:36:43.559 INFO:teuthology.orchestra.run.smithi092.stdout:checking gdbm-ndbm.h usability... no 2023-06-09T04:36:43.568 INFO:teuthology.orchestra.run.smithi092.stdout:checking gdbm-ndbm.h presence... no 2023-06-09T04:36:43.569 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gdbm-ndbm.h... no 2023-06-09T04:36:43.589 INFO:teuthology.orchestra.run.smithi092.stdout:checking gdbm/ndbm.h usability... no 2023-06-09T04:36:43.598 INFO:teuthology.orchestra.run.smithi092.stdout:checking gdbm/ndbm.h presence... no 2023-06-09T04:36:43.598 INFO:teuthology.orchestra.run.smithi092.stdout:checking for gdbm/ndbm.h... no 2023-06-09T04:36:43.619 INFO:teuthology.orchestra.run.smithi092.stdout:checking ndbm.h usability... no 2023-06-09T04:36:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:checking ndbm.h presence... no 2023-06-09T04:36:43.628 INFO:teuthology.orchestra.run.smithi092.stdout:checking for ndbm.h... no 2023-06-09T04:36:43.652 INFO:teuthology.orchestra.run.smithi092.stdout:checking libaio.h usability... yes 2023-06-09T04:36:43.662 INFO:teuthology.orchestra.run.smithi092.stdout:checking libaio.h presence... yes 2023-06-09T04:36:43.662 INFO:teuthology.orchestra.run.smithi092.stdout:checking for libaio.h... yes 2023-06-09T04:36:43.684 INFO:teuthology.orchestra.run.smithi092.stdout:checking liburing.h usability... no 2023-06-09T04:36:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:checking liburing.h presence... no 2023-06-09T04:36:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:checking for liburing.h... no 2023-06-09T04:36:43.714 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/capability.h usability... no 2023-06-09T04:36:43.723 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/capability.h presence... no 2023-06-09T04:36:43.723 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/capability.h... no 2023-06-09T04:36:43.752 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/fiemap.h usability... yes 2023-06-09T04:36:43.758 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/fiemap.h presence... yes 2023-06-09T04:36:43.759 INFO:teuthology.orchestra.run.smithi092.stdout:checking for linux/fiemap.h... yes 2023-06-09T04:36:43.790 INFO:teuthology.orchestra.run.smithi092.stdout:checking for fallocate... yes 2023-06-09T04:36:43.821 INFO:teuthology.orchestra.run.smithi092.stdout:checking for open_by_handle_at... yes 2023-06-09T04:36:43.844 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/prctl.h usability... yes 2023-06-09T04:36:43.851 INFO:teuthology.orchestra.run.smithi092.stdout:checking sys/prctl.h presence... yes 2023-06-09T04:36:43.852 INFO:teuthology.orchestra.run.smithi092.stdout:checking for sys/prctl.h... yes 2023-06-09T04:36:43.875 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/fs.h usability... yes 2023-06-09T04:36:43.881 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/fs.h presence... yes 2023-06-09T04:36:43.882 INFO:teuthology.orchestra.run.smithi092.stdout:checking for linux/fs.h... yes 2023-06-09T04:36:43.903 INFO:teuthology.orchestra.run.smithi092.stdout:checking btrfsutil.h usability... no 2023-06-09T04:36:43.912 INFO:teuthology.orchestra.run.smithi092.stdout:checking btrfsutil.h presence... no 2023-06-09T04:36:43.912 INFO:teuthology.orchestra.run.smithi092.stdout:checking for btrfsutil.h... no 2023-06-09T04:36:43.946 INFO:teuthology.orchestra.run.smithi092.stdout:checking for copy_file_range... yes 2023-06-09T04:36:43.980 INFO:teuthology.orchestra.run.smithi092.stdout:checking for SEEK_DATA... yes 2023-06-09T04:36:44.023 INFO:teuthology.orchestra.run.smithi092.stdout:checking for BMV_OF_SHARED... yes 2023-06-09T04:36:44.060 INFO:teuthology.orchestra.run.smithi092.stdout:checking for nftw... yes 2023-06-09T04:36:44.093 INFO:teuthology.orchestra.run.smithi092.stdout:checking for RLIMIT_NOFILE... yes 2023-06-09T04:36:44.110 INFO:teuthology.orchestra.run.smithi092.stdout:checking for FIEXCHANGE_RANGE... no 2023-06-09T04:36:44.141 INFO:teuthology.orchestra.run.smithi092.stdout:checking for renameat2... yes 2023-06-09T04:36:44.173 INFO:teuthology.orchestra.run.smithi092.stdout:checking for reallocarray... yes 2023-06-09T04:36:44.185 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct mount_attr... no 2023-06-09T04:36:44.233 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_qgroup_limit... yes 2023-06-09T04:36:44.265 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-06-09T04:36:44.299 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-06-09T04:36:44.332 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-06-09T04:36:44.364 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-06-09T04:36:44.397 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-06-09T04:36:44.431 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-06-09T04:36:44.457 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/btrfs.h usability... yes 2023-06-09T04:36:44.464 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/btrfs.h presence... yes 2023-06-09T04:36:44.465 INFO:teuthology.orchestra.run.smithi092.stdout:checking for linux/btrfs.h... yes 2023-06-09T04:36:44.493 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/btrfs_tree.h usability... yes 2023-06-09T04:36:44.500 INFO:teuthology.orchestra.run.smithi092.stdout:checking linux/btrfs_tree.h presence... yes 2023-06-09T04:36:44.501 INFO:teuthology.orchestra.run.smithi092.stdout:checking for linux/btrfs_tree.h... yes 2023-06-09T04:36:44.529 INFO:teuthology.orchestra.run.smithi092.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... no 2023-06-09T04:36:44.545 INFO:teuthology.orchestra.run.smithi092.stdout:configure: creating ./config.status 2023-06-09T04:36:44.836 INFO:teuthology.orchestra.run.smithi092.stdout:config.status: creating include/builddefs 2023-06-09T04:36:44.846 INFO:teuthology.orchestra.run.smithi092.stdout:config.status: creating include/config.h 2023-06-09T04:36:44.854 INFO:teuthology.orchestra.run.smithi092.stdout:config.status: executing libtool commands 2023-06-09T04:36:44.891 INFO:teuthology.orchestra.run.smithi092.stdout:Building include 2023-06-09T04:36:44.894 INFO:teuthology.orchestra.run.smithi092.stdout:Building lib 2023-06-09T04:36:44.897 INFO:teuthology.orchestra.run.smithi092.stdout: [LTDEP] 2023-06-09T04:36:44.999 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] dataascii.lo 2023-06-09T04:36:45.110 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] databin.lo 2023-06-09T04:36:45.246 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] datapid.lo 2023-06-09T04:36:45.341 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] file_lock.lo 2023-06-09T04:36:45.476 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] forker.lo 2023-06-09T04:36:45.599 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] pattern.lo 2023-06-09T04:36:45.712 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] open_flags.lo 2023-06-09T04:36:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] random_range.lo 2023-06-09T04:36:46.048 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] string_to_tokens.lo 2023-06-09T04:36:46.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:36:46.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:36:46.146 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] str_to_bytes.lo 2023-06-09T04:36:46.266 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] tlibio.lo 2023-06-09T04:36:46.475 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] write_log.lo 2023-06-09T04:36:46.692 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] random.lo 2023-06-09T04:36:46.797 INFO:teuthology.orchestra.run.smithi092.stdout: [LD] 2023-06-09T04:36:46.868 INFO:teuthology.orchestra.run.smithi092.stdout:Building ltp 2023-06-09T04:36:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: [DEP] 2023-06-09T04:36:46.977 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] doio 2023-06-09T04:36:47.564 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fsstress 2023-06-09T04:36:48.744 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fsx 2023-06-09T04:36:49.520 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] growfiles 2023-06-09T04:36:49.973 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] iogen 2023-06-09T04:36:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-stress 2023-06-09T04:36:50.664 INFO:teuthology.orchestra.run.smithi092.stdout:Building src 2023-06-09T04:36:50.669 INFO:teuthology.orchestra.run.smithi092.stdout: [DEP] 2023-06-09T04:36:51.935 INFO:teuthology.orchestra.run.smithi092.stdout: [DEP] 2023-06-09T04:36:51.960 INFO:teuthology.orchestra.run.smithi092.stdout: [DEP] 2023-06-09T04:36:51.967 INFO:teuthology.orchestra.run.smithi092.stdout: [DEP] 2023-06-09T04:36:52.146 INFO:teuthology.orchestra.run.smithi092.stdout: [DEP] 2023-06-09T04:36:52.167 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] dirstress 2023-06-09T04:36:52.378 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fill 2023-06-09T04:36:52.518 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fill2 2023-06-09T04:36:52.723 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] getpagesize 2023-06-09T04:36:52.848 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] holes 2023-06-09T04:36:53.023 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] lstat64 2023-06-09T04:36:53.189 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] nametest 2023-06-09T04:36:53.404 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] permname 2023-06-09T04:36:53.582 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] randholes 2023-06-09T04:36:53.804 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] runas 2023-06-09T04:36:53.962 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] truncfile 2023-06-09T04:36:54.124 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] usemem 2023-06-09T04:36:54.261 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] mmapcat 2023-06-09T04:36:54.397 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] append_reader 2023-06-09T04:36:54.533 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] append_writer 2023-06-09T04:36:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] dirperf 2023-06-09T04:36:54.865 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] metaperf 2023-06-09T04:36:55.093 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] devzero 2023-06-09T04:36:55.251 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] feature 2023-06-09T04:36:55.443 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] alloc 2023-06-09T04:36:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fault 2023-06-09T04:36:55.779 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fstest 2023-06-09T04:36:55.991 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_access_root 2023-06-09T04:36:56.132 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] godown 2023-06-09T04:36:56.285 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] resvtest 2023-06-09T04:36:56.450 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] writemod 2023-06-09T04:36:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] writev_on_pagefault 2023-06-09T04:36:56.727 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] makeextents 2023-06-09T04:36:56.887 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] itrash 2023-06-09T04:36:57.028 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] rename 2023-06-09T04:36:57.157 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] multi_open_unlink 2023-06-09T04:36:57.306 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] unwritten_sync 2023-06-09T04:36:57.468 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] genhashnames 2023-06-09T04:36:57.637 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_holes 2023-06-09T04:36:57.771 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_writev 2023-06-09T04:36:57.909 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_truncate_cmtime 2023-06-09T04:36:58.053 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] dirhash_collide 2023-06-09T04:36:58.217 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_rename_overwrite 2023-06-09T04:36:58.347 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] holetest 2023-06-09T04:36:58.557 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_truncate_self 2023-06-09T04:36:58.698 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] af_unix 2023-06-09T04:36:58.838 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_stale_pmd 2023-06-09T04:36:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_cow_race 2023-06-09T04:36:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_fallocate 2023-06-09T04:36:59.265 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fsync-err 2023-06-09T04:36:59.432 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_write_ro 2023-06-09T04:36:59.570 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_ext4_dax_journal_corruption 2023-06-09T04:36:59.713 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_ext4_dax_inline_corruption 2023-06-09T04:36:59.854 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_ofd_locks 2023-06-09T04:37:00.035 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_collision 2023-06-09T04:37:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] mmap-write-concurrent 2023-06-09T04:37:00.371 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_get_file_time 2023-06-09T04:37:00.506 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_create_short_dirs 2023-06-09T04:37:00.657 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_create_long_dirs 2023-06-09T04:37:00.806 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_enospc 2023-06-09T04:37:00.976 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_writev_overlap 2023-06-09T04:37:01.129 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] checkpoint_journal 2023-06-09T04:37:01.270 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] mmap-rw-fault 2023-06-09T04:37:01.433 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] allocstale 2023-06-09T04:37:01.583 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_cow_memory_failure 2023-06-09T04:37:01.736 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fake-dump-rootino 2023-06-09T04:37:01.873 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] dio-buf-fault 2023-06-09T04:37:02.014 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] xfsctl 2023-06-09T04:37:02.184 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] bstat 2023-06-09T04:37:02.367 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mtab 2023-06-09T04:37:02.534 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] getdevicesize 2023-06-09T04:37:02.664 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] preallo_rw_pattern_reader 2023-06-09T04:37:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] preallo_rw_pattern_writer 2023-06-09T04:37:02.951 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] ftrunc 2023-06-09T04:37:03.086 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] trunc 2023-06-09T04:37:03.234 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fs_perms 2023-06-09T04:37:03.383 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] testx 2023-06-09T04:37:03.504 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] looptest 2023-06-09T04:37:03.679 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] locktest 2023-06-09T04:37:03.954 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] unwritten_mmap 2023-06-09T04:37:04.117 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] bulkstat_unlink_test 2023-06-09T04:37:04.298 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] deduperace 2023-06-09T04:37:04.474 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] bulkstat_unlink_test_modified 2023-06-09T04:37:04.652 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_dir_offset 2023-06-09T04:37:04.787 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_futimens 2023-06-09T04:37:04.916 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_immutable 2023-06-09T04:37:05.458 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] stale_handle 2023-06-09T04:37:05.610 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] pwrite_mmap_blocked 2023-06-09T04:37:05.750 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_dir_offset2 2023-06-09T04:37:05.923 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] seek_sanity_test 2023-06-09T04:37:06.296 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] seek_copy_test 2023-06-09T04:37:06.457 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_readdir_1 2023-06-09T04:37:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_readdir_2 2023-06-09T04:37:06.729 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fsync-tester 2023-06-09T04:37:06.951 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] nsexec 2023-06-09T04:37:07.111 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] cloner 2023-06-09T04:37:07.272 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] renameat2 2023-06-09T04:37:07.422 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_getcwd 2023-06-09T04:37:07.566 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] e4compact 2023-06-09T04:37:07.747 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] test-nextquota 2023-06-09T04:37:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] punch-alternating 2023-06-09T04:37:08.055 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] attr-list-by-handle-cursor-test 2023-06-09T04:37:08.207 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] listxattr 2023-06-09T04:37:08.343 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] dio-interleaved 2023-06-09T04:37:08.489 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_dir_type 2023-06-09T04:37:08.631 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] dio-invalidate-cache 2023-06-09T04:37:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] stat_test 2023-06-09T04:37:09.082 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_encrypted_d_revalidate 2023-06-09T04:37:09.328 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] attr_replace_test 2023-06-09T04:37:09.476 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] swapon 2023-06-09T04:37:09.625 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] mkswap 2023-06-09T04:37:09.764 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_attr_corruption 2023-06-09T04:37:09.902 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_open_tmpfiles 2023-06-09T04:37:10.064 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fscrypt-crypt-util 2023-06-09T04:37:10.631 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] bulkstat_null_ocount 2023-06-09T04:37:10.775 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] splice-test 2023-06-09T04:37:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] chprojid_fail 2023-06-09T04:37:11.087 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] detached_mounts_propagation 2023-06-09T04:37:11.258 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] ext4_resize 2023-06-09T04:37:11.393 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_readdir_3 2023-06-09T04:37:11.569 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] splice2pipe 2023-06-09T04:37:11.718 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] uuid_ioctl 2023-06-09T04:37:11.860 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fiemap-tester 2023-06-09T04:37:12.100 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_stripealign 2023-06-09T04:37:12.242 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] open_by_handle 2023-06-09T04:37:12.471 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] t_mmap_dio 2023-06-09T04:37:12.619 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] xfsfind 2023-06-09T04:37:12.793 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] fssum 2023-06-09T04:37:13.181 INFO:teuthology.orchestra.run.smithi092.stdout:Building log-writes 2023-06-09T04:37:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] replay-log 2023-06-09T04:37:13.464 INFO:teuthology.orchestra.run.smithi092.stdout:Building perf 2023-06-09T04:37:13.467 INFO:teuthology.orchestra.run.smithi092.stdout:Building aio-dio-regress 2023-06-09T04:37:13.473 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-append-write-fallocate-race 2023-06-09T04:37:13.632 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-fcntl-race 2023-06-09T04:37:13.784 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-subblock-eof-read 2023-06-09T04:37:13.918 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-append-write-read-race 2023-06-09T04:37:14.080 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-invalidate-readahead 2023-06-09T04:37:14.233 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-write-verify 2023-06-09T04:37:14.409 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-06-09T04:37:14.536 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-last-ref-held-by-io 2023-06-09T04:37:14.696 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-cycle-write 2023-06-09T04:37:14.867 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-extend-stat 2023-06-09T04:37:15.019 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-06-09T04:37:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-hole-filling-race 2023-06-09T04:37:15.285 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-invalidate-failure 2023-06-09T04:37:15.434 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aiodio_sparse2 2023-06-09T04:37:15.630 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aiocp 2023-06-09T04:37:15.835 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-cow-race 2023-06-09T04:37:15.980 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] aio-dio-eof-race 2023-06-09T04:37:16.150 INFO:teuthology.orchestra.run.smithi092.stdout:Building vfs 2023-06-09T04:37:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] vfstest 2023-06-09T04:37:16.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:37:16.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:37:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: [CC] mount-idmapped 2023-06-09T04:37:18.149 INFO:teuthology.orchestra.run.smithi092.stdout:Building m4 2023-06-09T04:37:18.152 INFO:teuthology.orchestra.run.smithi092.stdout:Building common 2023-06-09T04:37:18.155 INFO:teuthology.orchestra.run.smithi092.stdout:Building tests 2023-06-09T04:37:18.161 INFO:teuthology.orchestra.run.smithi092.stdout:Building perf 2023-06-09T04:37:18.165 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/perf/group.list 2023-06-09T04:37:18.176 INFO:teuthology.orchestra.run.smithi092.stdout:Building ext4 2023-06-09T04:37:18.181 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ext4/group.list 2023-06-09T04:37:18.213 INFO:teuthology.orchestra.run.smithi092.stdout:Building ocfs2 2023-06-09T04:37:18.218 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ocfs2/group.list 2023-06-09T04:37:18.230 INFO:teuthology.orchestra.run.smithi092.stdout:Building overlay 2023-06-09T04:37:18.235 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/overlay/group.list 2023-06-09T04:37:18.269 INFO:teuthology.orchestra.run.smithi092.stdout:Building cifs 2023-06-09T04:37:18.273 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/cifs/group.list 2023-06-09T04:37:18.285 INFO:teuthology.orchestra.run.smithi092.stdout:Building generic 2023-06-09T04:37:18.291 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/generic/group.list 2023-06-09T04:37:18.378 INFO:teuthology.orchestra.run.smithi092.stdout:Building shared 2023-06-09T04:37:18.383 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/shared/group.list 2023-06-09T04:37:18.398 INFO:teuthology.orchestra.run.smithi092.stdout:Building nfs 2023-06-09T04:37:18.403 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/nfs/group.list 2023-06-09T04:37:18.416 INFO:teuthology.orchestra.run.smithi092.stdout:Building f2fs 2023-06-09T04:37:18.420 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/f2fs/group.list 2023-06-09T04:37:18.435 INFO:teuthology.orchestra.run.smithi092.stdout:Building udf 2023-06-09T04:37:18.440 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/udf/group.list 2023-06-09T04:37:18.450 INFO:teuthology.orchestra.run.smithi092.stdout:Building ceph 2023-06-09T04:37:18.455 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ceph/group.list 2023-06-09T04:37:18.468 INFO:teuthology.orchestra.run.smithi092.stdout:Building xfs 2023-06-09T04:37:18.474 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/xfs/group.list 2023-06-09T04:37:18.563 INFO:teuthology.orchestra.run.smithi092.stdout:Building btrfs 2023-06-09T04:37:18.569 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/btrfs/group.list 2023-06-09T04:37:18.621 INFO:teuthology.orchestra.run.smithi092.stdout:Building tmpfs 2023-06-09T04:37:18.625 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/tmpfs/group.list 2023-06-09T04:37:18.638 INFO:teuthology.orchestra.run.smithi092.stdout:Building selftest 2023-06-09T04:37:18.642 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/selftest/group.list 2023-06-09T04:37:18.655 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo make install) 2023-06-09T04:37:18.691 INFO:teuthology.orchestra.run.smithi092.stdout:Building include 2023-06-09T04:37:18.694 INFO:teuthology.orchestra.run.smithi092.stdout:Building lib 2023-06-09T04:37:18.698 INFO:teuthology.orchestra.run.smithi092.stdout:Building ltp 2023-06-09T04:37:18.702 INFO:teuthology.orchestra.run.smithi092.stdout:Building src 2023-06-09T04:37:18.734 INFO:teuthology.orchestra.run.smithi092.stdout:Building log-writes 2023-06-09T04:37:18.737 INFO:teuthology.orchestra.run.smithi092.stdout:Building perf 2023-06-09T04:37:18.740 INFO:teuthology.orchestra.run.smithi092.stdout:Building aio-dio-regress 2023-06-09T04:37:18.744 INFO:teuthology.orchestra.run.smithi092.stdout:Building vfs 2023-06-09T04:37:18.748 INFO:teuthology.orchestra.run.smithi092.stdout:Building m4 2023-06-09T04:37:18.751 INFO:teuthology.orchestra.run.smithi092.stdout:Building common 2023-06-09T04:37:18.754 INFO:teuthology.orchestra.run.smithi092.stdout:Building tests 2023-06-09T04:37:18.759 INFO:teuthology.orchestra.run.smithi092.stdout:Building perf 2023-06-09T04:37:18.764 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/perf/group.list 2023-06-09T04:37:18.774 INFO:teuthology.orchestra.run.smithi092.stdout:Building ext4 2023-06-09T04:37:18.779 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ext4/group.list 2023-06-09T04:37:18.812 INFO:teuthology.orchestra.run.smithi092.stdout:Building ocfs2 2023-06-09T04:37:18.817 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ocfs2/group.list 2023-06-09T04:37:18.829 INFO:teuthology.orchestra.run.smithi092.stdout:Building overlay 2023-06-09T04:37:18.834 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/overlay/group.list 2023-06-09T04:37:18.868 INFO:teuthology.orchestra.run.smithi092.stdout:Building cifs 2023-06-09T04:37:18.872 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/cifs/group.list 2023-06-09T04:37:18.883 INFO:teuthology.orchestra.run.smithi092.stdout:Building generic 2023-06-09T04:37:18.890 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/generic/group.list 2023-06-09T04:37:18.975 INFO:teuthology.orchestra.run.smithi092.stdout:Building shared 2023-06-09T04:37:18.980 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/shared/group.list 2023-06-09T04:37:18.995 INFO:teuthology.orchestra.run.smithi092.stdout:Building nfs 2023-06-09T04:37:19.000 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/nfs/group.list 2023-06-09T04:37:19.013 INFO:teuthology.orchestra.run.smithi092.stdout:Building f2fs 2023-06-09T04:37:19.018 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/f2fs/group.list 2023-06-09T04:37:19.033 INFO:teuthology.orchestra.run.smithi092.stdout:Building udf 2023-06-09T04:37:19.037 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/udf/group.list 2023-06-09T04:37:19.047 INFO:teuthology.orchestra.run.smithi092.stdout:Building ceph 2023-06-09T04:37:19.052 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ceph/group.list 2023-06-09T04:37:19.065 INFO:teuthology.orchestra.run.smithi092.stdout:Building xfs 2023-06-09T04:37:19.071 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/xfs/group.list 2023-06-09T04:37:19.158 INFO:teuthology.orchestra.run.smithi092.stdout:Building btrfs 2023-06-09T04:37:19.163 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/btrfs/group.list 2023-06-09T04:37:19.215 INFO:teuthology.orchestra.run.smithi092.stdout:Building tmpfs 2023-06-09T04:37:19.220 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/tmpfs/group.list 2023-06-09T04:37:19.232 INFO:teuthology.orchestra.run.smithi092.stdout:Building selftest 2023-06-09T04:37:19.236 INFO:teuthology.orchestra.run.smithi092.stdout: [GROUP] /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/selftest/group.list 2023-06-09T04:37:19.247 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2023-06-09T04:37:19.248 INFO:teuthology.orchestra.run.smithi092.stdout:gmake[1]: Nothing to be done for 'install'. 2023-06-09T04:37:19.249 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2023-06-09T04:37:19.251 INFO:teuthology.orchestra.run.smithi092.stdout:gmake[1]: Nothing to be done for 'install'. 2023-06-09T04:37:19.251 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2023-06-09T04:37:19.253 INFO:teuthology.orchestra.run.smithi092.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-06-09T04:37:19.261 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-06-09T04:37:19.337 INFO:teuthology.orchestra.run.smithi092.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-06-09T04:37:19.346 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2023-06-09T04:37:19.361 INFO:teuthology.orchestra.run.smithi092.stdout:Building log-writes 2023-06-09T04:37:19.365 INFO:teuthology.orchestra.run.smithi092.stdout:Building perf 2023-06-09T04:37:19.368 INFO:teuthology.orchestra.run.smithi092.stdout:Building aio-dio-regress 2023-06-09T04:37:19.372 INFO:teuthology.orchestra.run.smithi092.stdout:Building vfs 2023-06-09T04:37:19.374 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake -C log-writes install 2023-06-09T04:37:19.376 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-06-09T04:37:19.384 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-06-09T04:37:19.393 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake -C perf install 2023-06-09T04:37:19.394 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-06-09T04:37:19.403 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf 2023-06-09T04:37:19.425 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake -C aio-dio-regress install 2023-06-09T04:37:19.427 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-06-09T04:37:19.436 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 aio-dio-append-write-fallocate-race aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-hole-filling-race aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress 2023-06-09T04:37:19.488 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake -C vfs install 2023-06-09T04:37:19.489 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-06-09T04:37:19.498 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-06-09T04:37:19.510 INFO:teuthology.orchestra.run.smithi092.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-06-09T04:37:19.518 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-06-09T04:37:20.705 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py /var/lib/xfstests/src 2023-06-09T04:37:20.798 INFO:teuthology.orchestra.run.smithi092.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-06-09T04:37:20.829 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2023-06-09T04:37:20.830 INFO:teuthology.orchestra.run.smithi092.stdout:gmake[1]: Nothing to be done for 'install'. 2023-06-09T04:37:20.830 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2023-06-09T04:37:20.832 INFO:teuthology.orchestra.run.smithi092.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-06-09T04:37:20.840 INFO:teuthology.orchestra.run.smithi092.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-06-09T04:37:20.971 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2023-06-09T04:37:20.972 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/btrfs/ install 2023-06-09T04:37:20.975 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-06-09T04:37:20.983 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 290 291 299 /var/lib/xfstests/tests/btrfs 2023-06-09T04:37:21.768 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-06-09T04:37:21.776 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 057.out 175.out 186.out 240.out 069.out 092.out 077.out 140.out 265.out 236.out 108.out 204.out 177.out 194.out 234.out 249.out 109.out 033.out 082.out 252.out 270.out 040.out 176.out 202.out 137.out 034.out 125.out 014.out 165.out 181.out 152.out 142.out 264.out 192.out 148.out 282.out 267.out 052.out 228.out 183.out 094.out 273.out 251.out 110.out 043.out 209.out 151.out 206.out 195.out 274.out 215.out 115.out 015.out 250.out 191.out 281.out 046.out 051.out 244.out 219.out 119.out 019.out 128.out 232.out 248.out 162.out 147.out 197.out 196.out 242.out 004.out 189.out 220.out 120.out 020.out 102.out 263.out 171.out 261.out 045.out 217.out 117.out 017.out 072.out 068.out 048.out 062.out 038.out 084.out 129.out 233.out 066.out 029.out 229.out 103.out 254.out 111.out 011.out 243.out 275.out 134.out 170.out 260.out 157.out 143.out 133.out 286.out 161.out 246.out 216.out 116.out 016.out 155.out 118.out 287.out 067.out 010.out 238.out 138.out 095.out 184.out 160.out 190.out 291.out 081.out 024.out 071.out 061.out 047.out 200.out 100.out 025.out 139.out 258.out 185.out 269.out 208.out 039.out 222.out 035.out 169.out 098.out 290.out 080.out 070.out 060.out 207.out 106.out 009.out 167.out 003.out 211.out 158.out 085.out 146.out 188.out 104.out 089.out 086.out 279.out 225.out 174.out 156.out 135.out 257.out 124.out 172.out 058.out 262.out 036.out 042.out 193.out 097.out 271.out 283.out 083.out 199.out 099.out 278.out 053.out 164.out 239.out 241.out 002.out 178.out 141.out 214.out 114.out 074.out 056.out 012.out 041.out 223.out 150.out 123.out 179.out 144.out 032.out 277.out 205.out 136.out 005.out 096.out 198.out 008.out 280.out 050.out 105.out 027.out 187.out 091.out 153.out 088.out 285.out 044.out 201.out 101.out 001.out 266.out 237.out 065.out 231.out 227.out 075.out 131.out 213.out 113.out 013.out 031.out 182.out 272.out 078.out 173.out 259.out 210.out 132.out 037.out 093.out 235.out 127.out 230.out 166.out 130.out 023.out 073.out 030.out 203.out 063.out 055.out 245.out 145.out 159.out 122.out 076.out 022.out 006.out 284.out 299.out 224.out 154.out 276.out 149.out 064.out 218.out 018.out 168.out 007.out 079.out 054.out 059.out 049.out 226.out 163.out 126.out 121.out 026.out 180.out 268.out 253.out 255.out 090.out 221.out 087.out 212.out 112.out 256.out 021.out 247.out 028.out 107.out /var/lib/xfstests/tests/btrfs 2023-06-09T04:37:22.560 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ceph/ install 2023-06-09T04:37:22.562 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-06-09T04:37:22.570 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-06-09T04:37:22.589 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-06-09T04:37:22.598 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/ceph 2023-06-09T04:37:22.617 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/cifs/ install 2023-06-09T04:37:22.618 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-06-09T04:37:22.627 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-06-09T04:37:22.635 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-06-09T04:37:22.644 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-06-09T04:37:22.652 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ext4/ install 2023-06-09T04:37:22.654 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-06-09T04:37:22.663 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-06-09T04:37:22.851 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-06-09T04:37:22.859 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 001.out.nodelalloc 001.out.default 001.out.nozero 001.cfg 032.out 039.out 305.out 005.out 008.out 035.out 036.out 308.out 050.out 040.out 306.out 007.out 021.out 046.out 012.out 009.out 033.out 045.out 034.out 056.out 047.out 018.out 023.out 302.out 020.out 002.out 014.out 024.out 013.out 031.out 015.out 304.out 004.out 048.out 038.out 052.out 303.out 029.out 011.out 027.out 042.out 044.out 055.out 030.out 058.out 003.out 017.out 043.out 053.out 016.out 059.out 037.out 301.out 010.out 022.out 006.out 057.out 025.out 026.out 271.out 051.out 041.out 307.out 019.out 028.out 054.out 049.out /var/lib/xfstests/tests/ext4 2023-06-09T04:37:23.058 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/f2fs/ install 2023-06-09T04:37:23.059 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-06-09T04:37:23.068 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-06-09T04:37:23.079 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-06-09T04:37:23.088 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-06-09T04:37:23.099 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/generic/ install 2023-06-09T04:37:23.103 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-06-09T04:37:23.112 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 /var/lib/xfstests/tests/generic 2023-06-09T04:37:25.080 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-06-09T04:37:25.088 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 050.out.xfsquota 035.out.nfs 035.out.default 050.out.nojournal 050.out.default 050.cfg 035.cfg 208.out 433.out 670.out 040.out 104.out 575.out 537.out 683.out 497.out 337.out 702.out 460.out 149.out 088.out 366.out 202.out 137.out 034.out 110.out 389.out 403.out 593.out 571.out 243.out 592.out 282.out 472.out 468.out 628.out 528.out 607.out 428.out 659.out 407.out 228.out 128.out 171.out 028.out 620.out 273.out 555.out 492.out 609.out 707.out 693.out 595.out 506.out 268.out 632.out 532.out 591.out 281.out 471.out 619.out 255.out 013.out 419.out 319.out 177.out 523.out 119.out 019.out 727.out 372.out 645.out 673.out 190.out 686.out 312.out 718.out 143.out 716.out 161.out 516.out 518.out 538.out 655.out 216.out 116.out 016.out 155.out 338.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 457.out 333.out 258.out 362.out 379.out 691.out 039.out 173.out 498.out 665.out 098.out 711.out 290.out 480.out 706.out 060.out 406.out 306.out 106.out 642.out 447.out 217.out 120.out 191.out 411.out 481.out 211.out 158.out 014.out 182.out 370.out 188.out 030.out 099.out 585.out 724.out 624.out 524.out 174.out 156.out 324.out 224.out 313.out 355.out 124.out 672.out 278.out 063.out 058.out 384.out 296.out 042.out 193.out 271.out 465.out 473.out 335.out 247.out 509.out 568.out 309.out 441.out 440.out 475.out 548.out 356.out 341.out 094.out 241.out 600.out 089.out 714.out 614.out 514.out 414.out 314.out 416.out 214.out 381.out 342.out 041.out 664.out 340.out 358.out 572.out 586.out 521.out 240.out 549.out 095.out 633.out 434.out 140.out 708.out 533.out 508.out 408.out 599.out 476.out 087.out 199.out 458.out 233.out 639.out 721.out 540.out 397.out 053.out 279.out 276.out 448.out 164.out 376.out 589.out 587.out 339.out 294.out 352.out 484.out 489.out 139.out 317.out 399.out 220.out 552.out 695.out 017.out 677.out 175.out 315.out 295.out 580.out 075.out 186.out 452.out 679.out 292.out 252.out 077.out 167.out 688.out 206.out 348.out 394.out 651.out 184.out 466.out 697.out 495.out 725.out 109.out 482.out 625.out 283.out 525.out 488.out 496.out 325.out 551.out 225.out 322.out 636.out 125.out 715.out 192.out 544.out 515.out 650.out 152.out 658.out 563.out 393.out 578.out 451.out 015.out 392.out 682.out 684.out 550.out 052.out 652.out 534.out 351.out 583.out 450.out 463.out 251.out 267.out 176.out 069.out 644.out 065.out 487.out 615.out 395.out 676.out 415.out 344.out 215.out 115.out 444.out 027.out 250.out 391.out 674.out 464.out 705.out 536.out 577.out 439.out 244.out 336.out 144.out 505.out 405.out 057.out 205.out 136.out 598.out 363.out 036.out 198.out 723.out 008.out 390.out 680.out 605.out 305.out 105.out 005.out 367.out 291.out 719.out 323.out 685.out 562.out 382.out 570.out 388.out 527.out 237.out 151.out 227.out 163.out 044.out 332.out 661.out 582.out 232.out 478.out 584.out 462.out 597.out 412.out 132.out 179.out 037.out 293.out 499.out 638.out 353.out 020.out 660.out 561.out 141.out 076.out 668.out 648.out 387.out 594.out 438.out 265.out 289.out 574.out 006.out 690.out 284.out 519.out 567.out 299.out 474.out 456.out 461.out 477.out 307.out 647.out 493.out 107.out 007.out 380.out 526.out 479.out 426.out 150.out 326.out 082.out 226.out 413.out 126.out 025.out 121.out 026.out 021.out 485.out 694.out 347.out 275.out 090.out 361.out 564.out 302.out 432.out 002.out 558.out 162.out 147.out 288.out 197.out 033.out 396.out 051.out 350.out 547.out 047.out 024.out 360.out 543.out 261.out 045.out 692.out 443.out 726.out 378.out 346.out 048.out 062.out 626.out 529.out 231.out 343.out 662.out 425.out 329.out 165.out 334.out 103.out 003.out 129.out 423.out 371.out 234.out 183.out 671.out 486.out 223.out 123.out 262.out 669.out 023.out 459.out 616.out 032.out 266.out 146.out 398.out 696.out 427.out 590.out 280.out 470.out 455.out 187.out 091.out 046.out 374.out 631.out 328.out 260.out 654.out 359.out 531.out 649.out 301.out 101.out 635.out 623.out 385.out 331.out 556.out 713.out 613.out 513.out 622.out 131.out 213.out 634.out 031.out 565.out 630.out 675.out 272.out 530.out 510.out 627.out 573.out 576.out 430.out 310.out 569.out 454.out 259.out 210.out 449.out 327.out 330.out 067.out 093.out 604.out 287.out 127.out 699.out 166.out 130.out 219.out 073.out 603.out 503.out 303.out 203.out 354.out 157.out 386.out 349.out 055.out 245.out 709.out 522.out 422.out 222.out 159.out 539.out 249.out 122.out 689.out 022.out 681.out 424.out 321.out 066.out 490.out 435.out 375.out 134.out 154.out 566.out 618.out 501.out 418.out 318.out 542.out 218.out 118.out 018.out 377.out 172.out 168.out 079.out 469.out 054.out 059.out 049.out 442.out 657.out 621.out 180.out 270.out 253.out 421.out 712.out 612.out 667.out 512.out 494.out 221.out 212.out 112.out 256.out 012.out 029.out 195.out 554.out 148.out 185.out 596.out 720.out 248.out 196.out 520.out 242.out 420.out 285.out 320.out 629.out 230.out 602.out 502.out 402.out 102.out 717.out 617.out 517.out 429.out 417.out 365.out 092.out 117.out 357.out 072.out 068.out 229.out 038.out 084.out 056.out 369.out 611.out 511.out 311.out 368.out 637.out 111.out 011.out 257.out 194.out 170.out 606.out 133.out 286.out 043.out 437.out 545.out 559.out 710.out 610.out 687.out 410.out 701.out 601.out 401.out 201.out 010.out 001.out 096.out 114.out 113.out 074.out 581.out 145.out 641.out 491.out 081.out 071.out 083.out 653.out 700.out 064.out 400.out 300.out 200.out 100.out 646.out 541.out 445.out 269.out 546.out 656.out 698.out 553.out 178.out 298.out 446.out 080.out 070.out 608.out 507.out 409.out 308.out 209.out 108.out 009.out 643.out 004.out 557.out 453.out 666.out 236.out 078.out 364.out 097.out 383.out 085.out 704.out 588.out 504.out 404.out 304.out 204.out 535.out 345.out 246.out 467.out 086.out 254.out 207.out 142.out 235.out 436.out 703.out 135.out 264.out 431.out 678.out 373.out 316.out 169.out 640.out 579.out 297.out 153.out 560.out 663.out /var/lib/xfstests/tests/generic 2023-06-09T04:37:27.073 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/nfs/ install 2023-06-09T04:37:27.075 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-06-09T04:37:27.083 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-06-09T04:37:27.092 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-06-09T04:37:27.100 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-06-09T04:37:27.109 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/ocfs2/ install 2023-06-09T04:37:27.111 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-06-09T04:37:27.119 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-06-09T04:37:27.128 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-06-09T04:37:27.137 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-06-09T04:37:27.145 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/overlay/ install 2023-06-09T04:37:27.147 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-06-09T04:37:27.156 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-06-09T04:37:27.419 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-06-09T04:37:27.428 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 019.out 034.out 018.out 024.out 077.out 069.out 020.out 102.out 014.out 013.out 031.out 015.out 117.out 017.out 072.out 068.out 063.out 048.out 062.out 038.out 026.out 042.out 052.out 110.out 067.out 029.out 111.out 055.out 112.out 073.out 078.out 030.out 058.out 003.out 045.out 023.out 043.out 053.out 109.out 036.out 116.out 016.out 065.out 010.out 001.out 022.out 106.out 007.out 006.out 115.out 057.out 113.out 114.out 074.out 046.out 071.out 061.out 025.out 051.out 011.out 041.out 047.out 100.out 064.out 107.out 104.out 028.out 076.out 054.out 059.out 008.out 049.out 032.out 039.out 021.out 005.out 002.out 035.out 108.out 070.out 060.out 050.out 027.out 105.out 009.out 004.out 037.out 044.out 012.out 101.out 103.out 033.out 066.out 040.out 056.out 075.out /var/lib/xfstests/tests/overlay 2023-06-09T04:37:27.693 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/perf/ install 2023-06-09T04:37:27.694 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-06-09T04:37:27.703 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-06-09T04:37:27.711 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-06-09T04:37:27.720 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-06-09T04:37:27.729 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/selftest/ install 2023-06-09T04:37:27.731 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-06-09T04:37:27.739 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-06-09T04:37:27.761 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-06-09T04:37:27.770 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 006.out 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/selftest 2023-06-09T04:37:27.792 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/shared/ install 2023-06-09T04:37:27.793 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-06-09T04:37:27.802 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-06-09T04:37:27.815 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-06-09T04:37:27.824 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 032.out 298.out 002.out /var/lib/xfstests/tests/shared 2023-06-09T04:37:27.838 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/tmpfs/ install 2023-06-09T04:37:27.840 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-06-09T04:37:27.848 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-06-09T04:37:27.856 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-06-09T04:37:27.865 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-06-09T04:37:27.874 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/udf/ install 2023-06-09T04:37:27.875 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-06-09T04:37:27.883 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-06-09T04:37:27.892 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-06-09T04:37:27.901 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-06-09T04:37:27.909 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.3HuE1HNmdOxfstests-dev/tests/xfs/ install 2023-06-09T04:37:27.914 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-06-09T04:37:27.922 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-06-09T04:37:29.772 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-06-09T04:37:29.780 INFO:teuthology.orchestra.run.smithi092.stdout:../../install-sh -o root -g root -m 644 096.out.internal 033.out.crc 096.out.external 033.out.default 071.out.64 216.out.classic 216.out.64mblog 071.out.32 216.cfg 033.cfg 040.out 304.out 349.out 457.out 537.out 295.out 497.out 337.out 385.out 082.out 270.out 460.out 149.out 759.out 502.out 088.out 366.out 202.out 137.out 746.out 110.out 468.out 018.out 593.out 571.out 567.out 592.out 282.out 267.out 728.out 178.out 799.out 528.out 769.out 428.out 507.out 407.out 228.out 128.out 171.out 183.out 247.out 273.out 474.out 109.out 595.out 506.out 732.out 268.out 035.out 113.out 532.out 591.out 281.out 471.out 723.out 519.out 319.out 467.out 523.out 119.out 775.out 727.out 372.out 503.out 526.out 329.out 165.out 229.out 103.out 129.out 371.out 234.out 738.out 078.out 718.out 143.out 716.out 161.out 516.out 416.out 116.out 043.out 155.out 338.out 067.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 047.out 772.out 258.out 379.out 322.out 039.out 173.out 222.out 742.out 098.out 426.out 290.out 480.out 060.out 406.out 306.out 141.out 106.out 006.out 167.out 191.out 017.out 542.out 211.out 783.out 485.out 442.out 099.out 177.out 174.out 156.out 771.out 324.out 224.out 313.out 124.out 758.out 278.out 058.out 741.out 384.out 773.out 042.out 193.out 392.out 768.out 399.out 473.out 770.out 709.out 755.out 364.out 209.out 441.out 440.out 157.out 740.out 356.out 197.out 341.out 187.out 094.out 241.out 501.out 089.out 540.out 365.out 514.out 414.out 314.out 214.out 381.out 056.out 041.out 340.out 793.out 358.out 572.out 568.out 586.out 240.out 549.out 092.out 439.out 434.out 140.out 708.out 265.out 508.out 408.out 308.out 208.out 737.out 204.out 535.out 104.out 733.out 544.out 059.out 249.out 335.out 142.out 235.out 575.out 253.out 715.out 533.out 264.out 431.out 784.out 433.out 316.out 169.out 579.out 297.out 393.out 333.out 476.out 087.out 199.out 739.out 068.out 233.out 752.out 736.out 443.out 086.out 053.out 749.out 279.out 276.out 711.out 164.out 589.out 587.out 339.out 577.out 294.out 352.out 458.out 498.out 489.out 139.out 774.out 790.out 552.out 781.out 357.out 065.out 186.out 452.out 069.out 252.out 585.out 751.out 724.out 206.out 190.out 794.out 348.out 184.out 435.out 466.out 248.out 725.out 785.out 482.out 750.out 525.out 496.out 325.out 176.out 225.out 125.out 789.out 192.out 296.out 152.out 563.out 246.out 551.out 436.out 792.out 075.out 472.out 578.out 463.out 550.out 052.out 534.out 351.out 430.out 583.out 305.out 451.out 425.out 450.out 744.out 251.out 444.out 487.out 762.out 465.out 395.out 515.out 415.out 389.out 315.out 215.out 115.out 015.out 027.out 250.out 344.out 046.out 051.out 536.out 254.out 030.out 244.out 336.out 028.out 150.out 144.out 505.out 342.out 136.out 005.out 036.out 198.out 008.out 390.out 405.out 588.out 205.out 105.out 367.out 291.out 437.out 719.out 323.out 363.out 562.out 382.out 570.out 388.out 527.out 158.out 766.out 019.out 237.out 151.out 760.out 227.out 163.out 332.out 796.out 232.out 478.out 584.out 462.out 220.out 412.out 132.out 179.out 037.out 293.out 492.out 499.out 112.out 748.out 020.out 786.out 063.out 561.out 076.out 044.out 014.out 362.out 387.out 594.out 438.out 721.out 289.out 574.out 284.out 548.out 299.out 419.out 521.out 581.out 461.out 477.out 307.out 726.out 107.out 007.out 448.out 262.out 380.out 093.out 479.out 547.out 326.out 226.out 345.out 126.out 025.out 121.out 447.out 761.out 580.out 013.out 347.out 275.out 090.out 361.out 564.out 788.out 066.out 455.out 432.out 747.out 558.out 162.out 147.out 288.out 050.out 396.out 576.out 350.out 354.out 298.out 301.out 360.out 543.out 261.out 734.out 045.out 292.out 255.out 729.out 378.out 346.out 048.out 062.out 529.out 231.out 343.out 486.out 223.out 123.out 023.out 459.out 032.out 798.out 146.out 398.out 168.out 427.out 590.out 280.out 470.out 484.out 491.out 767.out 016.out 488.out 731.out 782.out 374.out 285.out 127.out 328.out 260.out 359.out 531.out 376.out 101.out 266.out 331.out 791.out 743.out 556.out 713.out 024.out 513.out 131.out 213.out 730.out 095.out 031.out 565.out 182.out 272.out 530.out 754.out 573.out 454.out 259.out 449.out 327.out 330.out 777.out 493.out 010.out 287.out 230.out 166.out 130.out 219.out 073.out 370.out 135.out 303.out 203.out 003.out 355.out 386.out 560.out 055.out 245.out 243.out 522.out 787.out 745.out 422.out 394.out 159.out 539.out 524.out 122.out 022.out 424.out 779.out 490.out 795.out 353.out 423.out 134.out 154.out 764.out 271.out 518.out 418.out 318.out 077.out 218.out 118.out 757.out 377.out 172.out 034.out 079.out 469.out 054.out 049.out 765.out 021.out 002.out 456.out 180.out 421.out 712.out 403.out 321.out 057.out 512.out 494.out 221.out 312.out 212.out 538.out 256.out 012.out 195.out 283.out 554.out 397.out 185.out 373.out 720.out 481.out 196.out 520.out 242.out 026.out 420.out 188.out 320.out 555.out 120.out 153.out 402.out 302.out 102.out 717.out 517.out 429.out 417.out 317.out 217.out 117.out 475.out 072.out 778.out 097.out 038.out 084.out 369.out 511.out 148.out 411.out 029.out 311.out 368.out 111.out 011.out 257.out 194.out 170.out 133.out 286.out 756.out 175.out 545.out 714.out 559.out 710.out 510.out 413.out 410.out 310.out 210.out 401.out 201.out 001.out 495.out 114.out 753.out 074.out 145.out 797.out 464.out 091.out 081.out 083.out 800.out 064.out 400.out 300.out 200.out 100.out 375.out 541.out 445.out 269.out 546.out 780.out 553.out 763.out 446.out 080.out 070.out 509.out 409.out 309.out 207.out 108.out 009.out 391.out 004.out 557.out 453.out 236.out 566.out 582.out 735.out 383.out 085.out 334.out 504.out 404.out 776.out /var/lib/xfstests/tests/xfs 2023-06-09T04:37:31.674 INFO:teuthology.orchestra.run.smithi092.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-06-09T04:37:31.682 INFO:teuthology.orchestra.run.smithi092.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-06-09T04:37:31.691 INFO:teuthology.orchestra.run.smithi092.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-06-09T04:37:31.700 INFO:teuthology.orchestra.run.smithi092.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-06-09T04:37:31.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:37:31.710 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec ls tests/generic) 2023-06-09T04:37:31.769 INFO:teuthology.orchestra.run.smithi092.stdout:001 2023-06-09T04:37:31.769 INFO:teuthology.orchestra.run.smithi092.stdout:001.out 2023-06-09T04:37:31.769 INFO:teuthology.orchestra.run.smithi092.stdout:002 2023-06-09T04:37:31.769 INFO:teuthology.orchestra.run.smithi092.stdout:002.out 2023-06-09T04:37:31.769 INFO:teuthology.orchestra.run.smithi092.stdout:003 2023-06-09T04:37:31.769 INFO:teuthology.orchestra.run.smithi092.stdout:003.out 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:004 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:004.out 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:005 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:005.out 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:006 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:006.out 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:007 2023-06-09T04:37:31.770 INFO:teuthology.orchestra.run.smithi092.stdout:007.out 2023-06-09T04:37:31.771 INFO:teuthology.orchestra.run.smithi092.stdout:008 2023-06-09T04:37:31.771 INFO:teuthology.orchestra.run.smithi092.stdout:008.out 2023-06-09T04:37:31.771 INFO:teuthology.orchestra.run.smithi092.stdout:009 2023-06-09T04:37:31.771 INFO:teuthology.orchestra.run.smithi092.stdout:009.out 2023-06-09T04:37:31.771 INFO:teuthology.orchestra.run.smithi092.stdout:010 2023-06-09T04:37:31.771 INFO:teuthology.orchestra.run.smithi092.stdout:010.out 2023-06-09T04:37:31.771 INFO:teuthology.orchestra.run.smithi092.stdout:011 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:011.out 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:012 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:012.out 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:013 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:013.out 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:014 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:014.out 2023-06-09T04:37:31.772 INFO:teuthology.orchestra.run.smithi092.stdout:015 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:015.out 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:016 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:016.out 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:017 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:017.out 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:018 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:018.out 2023-06-09T04:37:31.773 INFO:teuthology.orchestra.run.smithi092.stdout:019 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:019.out 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:020 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:020.out 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:021 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:021.out 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:022 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:022.out 2023-06-09T04:37:31.774 INFO:teuthology.orchestra.run.smithi092.stdout:023 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:023.out 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:024 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:024.out 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:025 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:025.out 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:026 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:026.out 2023-06-09T04:37:31.775 INFO:teuthology.orchestra.run.smithi092.stdout:027 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:027.out 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:028 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:028.out 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:029 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:029.out 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:030 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:030.out 2023-06-09T04:37:31.776 INFO:teuthology.orchestra.run.smithi092.stdout:031 2023-06-09T04:37:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:031.out 2023-06-09T04:37:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:032 2023-06-09T04:37:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:032.out 2023-06-09T04:37:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:033 2023-06-09T04:37:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:033.out 2023-06-09T04:37:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:034 2023-06-09T04:37:31.777 INFO:teuthology.orchestra.run.smithi092.stdout:034.out 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:035 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:035.cfg 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:035.out.default 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:035.out.nfs 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:036 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:036.out 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:037 2023-06-09T04:37:31.778 INFO:teuthology.orchestra.run.smithi092.stdout:037.out 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:038 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:038.out 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:039 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:039.out 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:040 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:040.out 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:041 2023-06-09T04:37:31.779 INFO:teuthology.orchestra.run.smithi092.stdout:041.out 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:042 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:042.out 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:043 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:043.out 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:044 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:044.out 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:045 2023-06-09T04:37:31.780 INFO:teuthology.orchestra.run.smithi092.stdout:045.out 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:046 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:046.out 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:047 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:047.out 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:048 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:048.out 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:049 2023-06-09T04:37:31.781 INFO:teuthology.orchestra.run.smithi092.stdout:049.out 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:050 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:050.cfg 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:050.out.default 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:050.out.nojournal 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:050.out.xfsquota 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:051 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:051.out 2023-06-09T04:37:31.782 INFO:teuthology.orchestra.run.smithi092.stdout:052 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:052.out 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:053 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:053.out 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:054 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:054.out 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:055 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:055.out 2023-06-09T04:37:31.783 INFO:teuthology.orchestra.run.smithi092.stdout:056 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:056.out 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:057 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:057.out 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:058 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:058.out 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:059 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:059.out 2023-06-09T04:37:31.784 INFO:teuthology.orchestra.run.smithi092.stdout:060 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:060.out 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:061 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:061.out 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:062 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:062.out 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:063 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:063.out 2023-06-09T04:37:31.785 INFO:teuthology.orchestra.run.smithi092.stdout:064 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:064.out 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:065 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:065.out 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:066 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:066.out 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:067 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:067.out 2023-06-09T04:37:31.786 INFO:teuthology.orchestra.run.smithi092.stdout:068 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:068.out 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:069 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:069.out 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:070 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:070.out 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:071 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:071.out 2023-06-09T04:37:31.787 INFO:teuthology.orchestra.run.smithi092.stdout:072 2023-06-09T04:37:31.788 INFO:teuthology.orchestra.run.smithi092.stdout:072.out 2023-06-09T04:37:31.788 INFO:teuthology.orchestra.run.smithi092.stdout:073 2023-06-09T04:37:31.788 INFO:teuthology.orchestra.run.smithi092.stdout:073.out 2023-06-09T04:37:31.788 INFO:teuthology.orchestra.run.smithi092.stdout:074 2023-06-09T04:37:31.788 INFO:teuthology.orchestra.run.smithi092.stdout:074.out 2023-06-09T04:37:31.788 INFO:teuthology.orchestra.run.smithi092.stdout:075 2023-06-09T04:37:31.788 INFO:teuthology.orchestra.run.smithi092.stdout:075.out 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:076 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:076.out 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:077 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:077.out 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:078 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:078.out 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:079 2023-06-09T04:37:31.789 INFO:teuthology.orchestra.run.smithi092.stdout:079.out 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:080 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:080.out 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:081 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:081.out 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:082 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:082.out 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:083 2023-06-09T04:37:31.790 INFO:teuthology.orchestra.run.smithi092.stdout:083.out 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:084 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:084.out 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:085 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:085.out 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:086 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:086.out 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:087 2023-06-09T04:37:31.791 INFO:teuthology.orchestra.run.smithi092.stdout:087.out 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:088 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:088.out 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:089 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:089.out 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:090 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:090.out 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:091 2023-06-09T04:37:31.792 INFO:teuthology.orchestra.run.smithi092.stdout:091.out 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:092 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:092.out 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:093 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:093.out 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:094 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:094.out 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:095 2023-06-09T04:37:31.793 INFO:teuthology.orchestra.run.smithi092.stdout:095.out 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:096 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:096.out 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:097 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:097.out 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:098 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:098.out 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:099 2023-06-09T04:37:31.794 INFO:teuthology.orchestra.run.smithi092.stdout:099.out 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:100 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:100.out 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:101 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:101.out 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:102 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:102.out 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:103 2023-06-09T04:37:31.795 INFO:teuthology.orchestra.run.smithi092.stdout:103.out 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:104 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:104.out 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:105 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:105.out 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:106 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:106.out 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:107 2023-06-09T04:37:31.796 INFO:teuthology.orchestra.run.smithi092.stdout:107.out 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:108 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:108.out 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:109 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:109.out 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:110 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:110.out 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:111 2023-06-09T04:37:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:111.out 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:112 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:112.out 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:113 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:113.out 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:114 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:114.out 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:115 2023-06-09T04:37:31.798 INFO:teuthology.orchestra.run.smithi092.stdout:115.out 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:116 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:116.out 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:117 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:117.out 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:118 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:118.out 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:119 2023-06-09T04:37:31.799 INFO:teuthology.orchestra.run.smithi092.stdout:119.out 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:120 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:120.out 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:121 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:121.out 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:122 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:122.out 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:123 2023-06-09T04:37:31.800 INFO:teuthology.orchestra.run.smithi092.stdout:123.out 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:124 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:124.out 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:125 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:125.out 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:126 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:126.out 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:127 2023-06-09T04:37:31.801 INFO:teuthology.orchestra.run.smithi092.stdout:127.out 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:128 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:128.out 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:129 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:129.out 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:130 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:130.out 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:131 2023-06-09T04:37:31.802 INFO:teuthology.orchestra.run.smithi092.stdout:131.out 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:132 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:132.out 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:133 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:133.out 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:134 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:134.out 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:135 2023-06-09T04:37:31.803 INFO:teuthology.orchestra.run.smithi092.stdout:135.out 2023-06-09T04:37:31.804 INFO:teuthology.orchestra.run.smithi092.stdout:136 2023-06-09T04:37:31.804 INFO:teuthology.orchestra.run.smithi092.stdout:136.out 2023-06-09T04:37:31.804 INFO:teuthology.orchestra.run.smithi092.stdout:137 2023-06-09T04:37:31.804 INFO:teuthology.orchestra.run.smithi092.stdout:137.out 2023-06-09T04:37:31.804 INFO:teuthology.orchestra.run.smithi092.stdout:138 2023-06-09T04:37:31.804 INFO:teuthology.orchestra.run.smithi092.stdout:138.out 2023-06-09T04:37:31.804 INFO:teuthology.orchestra.run.smithi092.stdout:139 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:139.out 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:140 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:140.out 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:141 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:141.out 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:142 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:142.out 2023-06-09T04:37:31.805 INFO:teuthology.orchestra.run.smithi092.stdout:143 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:143.out 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:144 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:144.out 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:145 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:145.out 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:146 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:146.out 2023-06-09T04:37:31.806 INFO:teuthology.orchestra.run.smithi092.stdout:147 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:147.out 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:148 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:148.out 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:149 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:149.out 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:150 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:150.out 2023-06-09T04:37:31.807 INFO:teuthology.orchestra.run.smithi092.stdout:151 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:151.out 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:152 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:152.out 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:153 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:153.out 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:154 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:154.out 2023-06-09T04:37:31.808 INFO:teuthology.orchestra.run.smithi092.stdout:155 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:155.out 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:156 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:156.out 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:157 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:157.out 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:158 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:158.out 2023-06-09T04:37:31.809 INFO:teuthology.orchestra.run.smithi092.stdout:159 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:159.out 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:160 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:160.out 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:161 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:161.out 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:162 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:162.out 2023-06-09T04:37:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:163 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:163.out 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:164 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:164.out 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:165 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:165.out 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:166 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:166.out 2023-06-09T04:37:31.811 INFO:teuthology.orchestra.run.smithi092.stdout:167 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:167.out 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:168 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:168.out 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:169 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:169.out 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:170 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:170.out 2023-06-09T04:37:31.812 INFO:teuthology.orchestra.run.smithi092.stdout:171 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:171.out 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:172 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:172.out 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:173 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:173.out 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:174 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:174.out 2023-06-09T04:37:31.813 INFO:teuthology.orchestra.run.smithi092.stdout:175 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:175.out 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:176 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:176.out 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:177 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:177.out 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:178 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:178.out 2023-06-09T04:37:31.814 INFO:teuthology.orchestra.run.smithi092.stdout:179 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:179.out 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:180 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:180.out 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:181 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:181.out 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:182 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:182.out 2023-06-09T04:37:31.815 INFO:teuthology.orchestra.run.smithi092.stdout:183 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:183.out 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:184 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:184.out 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:185 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:185.out 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:186 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:186.out 2023-06-09T04:37:31.816 INFO:teuthology.orchestra.run.smithi092.stdout:187 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:187.out 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:188 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:188.out 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:189 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:189.out 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:190 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:190.out 2023-06-09T04:37:31.817 INFO:teuthology.orchestra.run.smithi092.stdout:191 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:191.out 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:192 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:192.out 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:193 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:193.out 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:194 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:194.out 2023-06-09T04:37:31.818 INFO:teuthology.orchestra.run.smithi092.stdout:195 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:195.out 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:196 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:196.out 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:197 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:197.out 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:198 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:198.out 2023-06-09T04:37:31.819 INFO:teuthology.orchestra.run.smithi092.stdout:199 2023-06-09T04:37:31.820 INFO:teuthology.orchestra.run.smithi092.stdout:199.out 2023-06-09T04:37:31.820 INFO:teuthology.orchestra.run.smithi092.stdout:200 2023-06-09T04:37:31.820 INFO:teuthology.orchestra.run.smithi092.stdout:200.out 2023-06-09T04:37:31.820 INFO:teuthology.orchestra.run.smithi092.stdout:201 2023-06-09T04:37:31.820 INFO:teuthology.orchestra.run.smithi092.stdout:201.out 2023-06-09T04:37:31.820 INFO:teuthology.orchestra.run.smithi092.stdout:202 2023-06-09T04:37:31.820 INFO:teuthology.orchestra.run.smithi092.stdout:202.out 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:203 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:203.out 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:204 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:204.out 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:205 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:205.out 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:206 2023-06-09T04:37:31.821 INFO:teuthology.orchestra.run.smithi092.stdout:206.out 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:207 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:207.out 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:208 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:208.out 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:209 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:209.out 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:210 2023-06-09T04:37:31.822 INFO:teuthology.orchestra.run.smithi092.stdout:210.out 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:211 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:211.out 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:212 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:212.out 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:213 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:213.out 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:214 2023-06-09T04:37:31.823 INFO:teuthology.orchestra.run.smithi092.stdout:214.out 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:215 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:215.out 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:216 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:216.out 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:217 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:217.out 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:218 2023-06-09T04:37:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:218.out 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:219 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:219.out 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:220 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:220.out 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:221 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:221.out 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:222 2023-06-09T04:37:31.825 INFO:teuthology.orchestra.run.smithi092.stdout:222.out 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:223 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:223.out 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:224 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:224.out 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:225 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:225.out 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:226 2023-06-09T04:37:31.826 INFO:teuthology.orchestra.run.smithi092.stdout:226.out 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:227 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:227.out 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:228 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:228.out 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:229 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:229.out 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:230 2023-06-09T04:37:31.827 INFO:teuthology.orchestra.run.smithi092.stdout:230.out 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:231 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:231.out 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:232 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:232.out 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:233 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:233.out 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:234 2023-06-09T04:37:31.828 INFO:teuthology.orchestra.run.smithi092.stdout:234.out 2023-06-09T04:37:31.829 INFO:teuthology.orchestra.run.smithi092.stdout:235 2023-06-09T04:37:31.829 INFO:teuthology.orchestra.run.smithi092.stdout:235.out 2023-06-09T04:37:31.829 INFO:teuthology.orchestra.run.smithi092.stdout:236 2023-06-09T04:37:31.829 INFO:teuthology.orchestra.run.smithi092.stdout:236.out 2023-06-09T04:37:31.829 INFO:teuthology.orchestra.run.smithi092.stdout:237 2023-06-09T04:37:31.829 INFO:teuthology.orchestra.run.smithi092.stdout:237.out 2023-06-09T04:37:31.829 INFO:teuthology.orchestra.run.smithi092.stdout:238 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:238.out 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:239 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:239.out 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:240 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:240.out 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:241 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:241.out 2023-06-09T04:37:31.830 INFO:teuthology.orchestra.run.smithi092.stdout:242 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:242.out 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:243 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:243.out 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:244 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:244.out 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:245 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:245.out 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:246 2023-06-09T04:37:31.831 INFO:teuthology.orchestra.run.smithi092.stdout:246.out 2023-06-09T04:37:31.832 INFO:teuthology.orchestra.run.smithi092.stdout:247 2023-06-09T04:37:31.832 INFO:teuthology.orchestra.run.smithi092.stdout:247.out 2023-06-09T04:37:31.832 INFO:teuthology.orchestra.run.smithi092.stdout:248 2023-06-09T04:37:31.832 INFO:teuthology.orchestra.run.smithi092.stdout:248.out 2023-06-09T04:37:31.832 INFO:teuthology.orchestra.run.smithi092.stdout:249 2023-06-09T04:37:31.832 INFO:teuthology.orchestra.run.smithi092.stdout:249.out 2023-06-09T04:37:31.832 INFO:teuthology.orchestra.run.smithi092.stdout:250 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:250.out 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:251 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:251.out 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:252 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:252.out 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:253 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:253.out 2023-06-09T04:37:31.833 INFO:teuthology.orchestra.run.smithi092.stdout:254 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:254.out 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:255 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:255.out 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:256 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:256.out 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:257 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:257.out 2023-06-09T04:37:31.834 INFO:teuthology.orchestra.run.smithi092.stdout:258 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:258.out 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:259 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:259.out 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:260 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:260.out 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:261 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:261.out 2023-06-09T04:37:31.835 INFO:teuthology.orchestra.run.smithi092.stdout:262 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:262.out 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:263 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:263.out 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:264 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:264.out 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:265 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:265.out 2023-06-09T04:37:31.836 INFO:teuthology.orchestra.run.smithi092.stdout:266 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:266.out 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:267 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:267.out 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:268 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:268.out 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:269 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:269.out 2023-06-09T04:37:31.837 INFO:teuthology.orchestra.run.smithi092.stdout:270 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:270.out 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:271 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:271.out 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:272 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:272.out 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:273 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:273.out 2023-06-09T04:37:31.838 INFO:teuthology.orchestra.run.smithi092.stdout:274 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:274.out 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:275 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:275.out 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:276 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:276.out 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:277 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:277.out 2023-06-09T04:37:31.839 INFO:teuthology.orchestra.run.smithi092.stdout:278 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:278.out 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:279 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:279.out 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:280 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:280.out 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:281 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:281.out 2023-06-09T04:37:31.840 INFO:teuthology.orchestra.run.smithi092.stdout:282 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:282.out 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:283 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:283.out 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:284 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:284.out 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:285 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:285.out 2023-06-09T04:37:31.841 INFO:teuthology.orchestra.run.smithi092.stdout:286 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:286.out 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:287 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:287.out 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:288 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:288.out 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:289 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:289.out 2023-06-09T04:37:31.842 INFO:teuthology.orchestra.run.smithi092.stdout:290 2023-06-09T04:37:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:290.out 2023-06-09T04:37:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:291 2023-06-09T04:37:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:291.out 2023-06-09T04:37:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:292 2023-06-09T04:37:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:292.out 2023-06-09T04:37:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:293 2023-06-09T04:37:31.843 INFO:teuthology.orchestra.run.smithi092.stdout:293.out 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:294 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:294.out 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:295 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:295.out 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:296 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:296.out 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:297 2023-06-09T04:37:31.844 INFO:teuthology.orchestra.run.smithi092.stdout:297.out 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:298 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:298.out 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:299 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:299.out 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:300 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:300.out 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:301 2023-06-09T04:37:31.845 INFO:teuthology.orchestra.run.smithi092.stdout:301.out 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:302 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:302.out 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:303 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:303.out 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:304 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:304.out 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:305 2023-06-09T04:37:31.846 INFO:teuthology.orchestra.run.smithi092.stdout:305.out 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:306 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:306.out 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:307 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:307.out 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:308 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:308.out 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:309 2023-06-09T04:37:31.847 INFO:teuthology.orchestra.run.smithi092.stdout:309.out 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:310 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:310.out 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:311 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:311.out 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:312 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:312.out 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:313 2023-06-09T04:37:31.848 INFO:teuthology.orchestra.run.smithi092.stdout:313.out 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:314 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:314.out 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:315 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:315.out 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:316 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:316.out 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:317 2023-06-09T04:37:31.849 INFO:teuthology.orchestra.run.smithi092.stdout:317.out 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:318 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:318.out 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:319 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:319.out 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:320 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:320.out 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:321 2023-06-09T04:37:31.850 INFO:teuthology.orchestra.run.smithi092.stdout:321.out 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:322 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:322.out 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:323 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:323.out 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:324 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:324.out 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:325 2023-06-09T04:37:31.851 INFO:teuthology.orchestra.run.smithi092.stdout:325.out 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:326 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:326.out 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:327 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:327.out 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:328 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:328.out 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:329 2023-06-09T04:37:31.852 INFO:teuthology.orchestra.run.smithi092.stdout:329.out 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:330 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:330.out 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:331 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:331.out 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:332 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:332.out 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:333 2023-06-09T04:37:31.853 INFO:teuthology.orchestra.run.smithi092.stdout:333.out 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:334 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:334.out 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:335 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:335.out 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:336 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:336.out 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:337 2023-06-09T04:37:31.854 INFO:teuthology.orchestra.run.smithi092.stdout:337.out 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:338 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:338.out 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:339 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:339.out 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:340 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:340.out 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:341 2023-06-09T04:37:31.855 INFO:teuthology.orchestra.run.smithi092.stdout:341.out 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:342 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:342.out 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:343 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:343.out 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:344 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:344.out 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:345 2023-06-09T04:37:31.856 INFO:teuthology.orchestra.run.smithi092.stdout:345.out 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:346 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:346.out 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:347 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:347.out 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:348 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:348.out 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:349 2023-06-09T04:37:31.857 INFO:teuthology.orchestra.run.smithi092.stdout:349.out 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:350 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:350.out 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:351 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:351.out 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:352 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:352.out 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:353 2023-06-09T04:37:31.858 INFO:teuthology.orchestra.run.smithi092.stdout:353.out 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:354 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:354.out 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:355 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:355.out 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:356 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:356.out 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:357 2023-06-09T04:37:31.859 INFO:teuthology.orchestra.run.smithi092.stdout:357.out 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:358 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:358.out 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:359 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:359.out 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:360 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:360.out 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:361 2023-06-09T04:37:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:361.out 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:362 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:362.out 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:363 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:363.out 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:364 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:364.out 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:365 2023-06-09T04:37:31.861 INFO:teuthology.orchestra.run.smithi092.stdout:365.out 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:366 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:366.out 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:367 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:367.out 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:368 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:368.out 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:369 2023-06-09T04:37:31.862 INFO:teuthology.orchestra.run.smithi092.stdout:369.out 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:370 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:370.out 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:371 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:371.out 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:372 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:372.out 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:373 2023-06-09T04:37:31.863 INFO:teuthology.orchestra.run.smithi092.stdout:373.out 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:374 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:374.out 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:375 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:375.out 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:376 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:376.out 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:377 2023-06-09T04:37:31.864 INFO:teuthology.orchestra.run.smithi092.stdout:377.out 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:378 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:378.out 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:379 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:379.out 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:380 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:380.out 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:381 2023-06-09T04:37:31.865 INFO:teuthology.orchestra.run.smithi092.stdout:381.out 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:382 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:382.out 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:383 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:383.out 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:384 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:384.out 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:385 2023-06-09T04:37:31.866 INFO:teuthology.orchestra.run.smithi092.stdout:385.out 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:386 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:386.out 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:387 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:387.out 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:388 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:388.out 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:389 2023-06-09T04:37:31.867 INFO:teuthology.orchestra.run.smithi092.stdout:389.out 2023-06-09T04:37:31.868 INFO:teuthology.orchestra.run.smithi092.stdout:390 2023-06-09T04:37:31.868 INFO:teuthology.orchestra.run.smithi092.stdout:390.out 2023-06-09T04:37:31.868 INFO:teuthology.orchestra.run.smithi092.stdout:391 2023-06-09T04:37:31.868 INFO:teuthology.orchestra.run.smithi092.stdout:391.out 2023-06-09T04:37:31.868 INFO:teuthology.orchestra.run.smithi092.stdout:392 2023-06-09T04:37:31.868 INFO:teuthology.orchestra.run.smithi092.stdout:392.out 2023-06-09T04:37:31.868 INFO:teuthology.orchestra.run.smithi092.stdout:393 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:393.out 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:394 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:394.out 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:395 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:395.out 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:396 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:396.out 2023-06-09T04:37:31.869 INFO:teuthology.orchestra.run.smithi092.stdout:397 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:397.out 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:398 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:398.out 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:399 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:399.out 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:400 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:400.out 2023-06-09T04:37:31.870 INFO:teuthology.orchestra.run.smithi092.stdout:401 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:401.out 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:402 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:402.out 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:403 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:403.out 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:404 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:404.out 2023-06-09T04:37:31.871 INFO:teuthology.orchestra.run.smithi092.stdout:405 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:405.out 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:406 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:406.out 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:407 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:407.out 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:408 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:408.out 2023-06-09T04:37:31.872 INFO:teuthology.orchestra.run.smithi092.stdout:409 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:409.out 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:410 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:410.out 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:411 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:411.out 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:412 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:412.out 2023-06-09T04:37:31.873 INFO:teuthology.orchestra.run.smithi092.stdout:413 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:413.out 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:414 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:414.out 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:415 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:415.out 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:416 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:416.out 2023-06-09T04:37:31.874 INFO:teuthology.orchestra.run.smithi092.stdout:417 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:417.out 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:418 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:418.out 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:419 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:419.out 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:420 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:420.out 2023-06-09T04:37:31.875 INFO:teuthology.orchestra.run.smithi092.stdout:421 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:421.out 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:422 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:422.out 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:423 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:423.out 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:424 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:424.out 2023-06-09T04:37:31.876 INFO:teuthology.orchestra.run.smithi092.stdout:425 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:425.out 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:426 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:426.out 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:427 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:427.out 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:428 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:428.out 2023-06-09T04:37:31.877 INFO:teuthology.orchestra.run.smithi092.stdout:429 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:429.out 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:430 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:430.out 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:431 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:431.out 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:432 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:432.out 2023-06-09T04:37:31.878 INFO:teuthology.orchestra.run.smithi092.stdout:433 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:433.out 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:434 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:434.out 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:435 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:435.out 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:436 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:436.out 2023-06-09T04:37:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:437 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:437.out 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:438 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:438.out 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:439 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:439.out 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:440 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:440.out 2023-06-09T04:37:31.880 INFO:teuthology.orchestra.run.smithi092.stdout:441 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:441.out 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:442 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:442.out 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:443 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:443.out 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:444 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:444.out 2023-06-09T04:37:31.881 INFO:teuthology.orchestra.run.smithi092.stdout:445 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:445.out 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:446 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:446.out 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:447 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:447.out 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:448 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:448.out 2023-06-09T04:37:31.882 INFO:teuthology.orchestra.run.smithi092.stdout:449 2023-06-09T04:37:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:449.out 2023-06-09T04:37:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:450 2023-06-09T04:37:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:450.out 2023-06-09T04:37:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:451 2023-06-09T04:37:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:451.out 2023-06-09T04:37:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:452 2023-06-09T04:37:31.883 INFO:teuthology.orchestra.run.smithi092.stdout:452.out 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:453 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:453.out 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:454 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:454.out 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:455 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:455.out 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:456 2023-06-09T04:37:31.884 INFO:teuthology.orchestra.run.smithi092.stdout:456.out 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:457 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:457.out 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:458 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:458.out 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:459 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:459.out 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:460 2023-06-09T04:37:31.885 INFO:teuthology.orchestra.run.smithi092.stdout:460.out 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:461 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:461.out 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:462 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:462.out 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:463 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:463.out 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:464 2023-06-09T04:37:31.886 INFO:teuthology.orchestra.run.smithi092.stdout:464.out 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:465 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:465.out 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:466 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:466.out 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:467 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:467.out 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:468 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:468.out 2023-06-09T04:37:31.887 INFO:teuthology.orchestra.run.smithi092.stdout:469 2023-06-09T04:37:31.888 INFO:teuthology.orchestra.run.smithi092.stdout:469.out 2023-06-09T04:37:31.888 INFO:teuthology.orchestra.run.smithi092.stdout:470 2023-06-09T04:37:31.888 INFO:teuthology.orchestra.run.smithi092.stdout:470.out 2023-06-09T04:37:31.888 INFO:teuthology.orchestra.run.smithi092.stdout:471 2023-06-09T04:37:31.888 INFO:teuthology.orchestra.run.smithi092.stdout:471.out 2023-06-09T04:37:31.888 INFO:teuthology.orchestra.run.smithi092.stdout:472 2023-06-09T04:37:31.888 INFO:teuthology.orchestra.run.smithi092.stdout:472.out 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:473 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:473.out 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:474 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:474.out 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:475 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:475.out 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:476 2023-06-09T04:37:31.889 INFO:teuthology.orchestra.run.smithi092.stdout:476.out 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:477 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:477.out 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:478 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:478.out 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:479 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:479.out 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:480 2023-06-09T04:37:31.890 INFO:teuthology.orchestra.run.smithi092.stdout:480.out 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:481 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:481.out 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:482 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:482.out 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:483 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:483.out 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:484 2023-06-09T04:37:31.891 INFO:teuthology.orchestra.run.smithi092.stdout:484.out 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:485 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:485.out 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:486 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:486.out 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:487 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:487.out 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:488 2023-06-09T04:37:31.892 INFO:teuthology.orchestra.run.smithi092.stdout:488.out 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:489 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:489.out 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:490 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:490.out 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:491 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:491.out 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:492 2023-06-09T04:37:31.893 INFO:teuthology.orchestra.run.smithi092.stdout:492.out 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:493 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:493.out 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:494 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:494.out 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:495 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:495.out 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:496 2023-06-09T04:37:31.894 INFO:teuthology.orchestra.run.smithi092.stdout:496.out 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:497 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:497.out 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:498 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:498.out 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:499 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:499.out 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:500 2023-06-09T04:37:31.895 INFO:teuthology.orchestra.run.smithi092.stdout:500.out 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:501 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:501.out 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:502 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:502.out 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:503 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:503.out 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:504 2023-06-09T04:37:31.896 INFO:teuthology.orchestra.run.smithi092.stdout:504.out 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:505 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:505.out 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:506 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:506.out 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:507 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:507.out 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:508 2023-06-09T04:37:31.897 INFO:teuthology.orchestra.run.smithi092.stdout:508.out 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:509 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:509.out 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:510 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:510.out 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:511 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:511.out 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:512 2023-06-09T04:37:31.898 INFO:teuthology.orchestra.run.smithi092.stdout:512.out 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:513 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:513.out 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:514 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:514.out 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:515 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:515.out 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:516 2023-06-09T04:37:31.899 INFO:teuthology.orchestra.run.smithi092.stdout:516.out 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:517 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:517.out 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:518 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:518.out 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:519 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:519.out 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:520 2023-06-09T04:37:31.900 INFO:teuthology.orchestra.run.smithi092.stdout:520.out 2023-06-09T04:37:31.901 INFO:teuthology.orchestra.run.smithi092.stdout:521 2023-06-09T04:37:31.901 INFO:teuthology.orchestra.run.smithi092.stdout:521.out 2023-06-09T04:37:31.901 INFO:teuthology.orchestra.run.smithi092.stdout:522 2023-06-09T04:37:31.901 INFO:teuthology.orchestra.run.smithi092.stdout:522.out 2023-06-09T04:37:31.901 INFO:teuthology.orchestra.run.smithi092.stdout:523 2023-06-09T04:37:31.902 INFO:teuthology.orchestra.run.smithi092.stdout:523.out 2023-06-09T04:37:31.902 INFO:teuthology.orchestra.run.smithi092.stdout:524 2023-06-09T04:37:31.902 INFO:teuthology.orchestra.run.smithi092.stdout:524.out 2023-06-09T04:37:31.902 INFO:teuthology.orchestra.run.smithi092.stdout:525 2023-06-09T04:37:31.902 INFO:teuthology.orchestra.run.smithi092.stdout:525.out 2023-06-09T04:37:31.902 INFO:teuthology.orchestra.run.smithi092.stdout:526 2023-06-09T04:37:31.903 INFO:teuthology.orchestra.run.smithi092.stdout:526.out 2023-06-09T04:37:31.903 INFO:teuthology.orchestra.run.smithi092.stdout:527 2023-06-09T04:37:31.903 INFO:teuthology.orchestra.run.smithi092.stdout:527.out 2023-06-09T04:37:31.903 INFO:teuthology.orchestra.run.smithi092.stdout:528 2023-06-09T04:37:31.904 INFO:teuthology.orchestra.run.smithi092.stdout:528.out 2023-06-09T04:37:31.904 INFO:teuthology.orchestra.run.smithi092.stdout:529 2023-06-09T04:37:31.904 INFO:teuthology.orchestra.run.smithi092.stdout:529.out 2023-06-09T04:37:31.905 INFO:teuthology.orchestra.run.smithi092.stdout:530 2023-06-09T04:37:31.905 INFO:teuthology.orchestra.run.smithi092.stdout:530.out 2023-06-09T04:37:31.905 INFO:teuthology.orchestra.run.smithi092.stdout:531 2023-06-09T04:37:31.905 INFO:teuthology.orchestra.run.smithi092.stdout:531.out 2023-06-09T04:37:31.906 INFO:teuthology.orchestra.run.smithi092.stdout:532 2023-06-09T04:37:31.906 INFO:teuthology.orchestra.run.smithi092.stdout:532.out 2023-06-09T04:37:31.906 INFO:teuthology.orchestra.run.smithi092.stdout:533 2023-06-09T04:37:31.907 INFO:teuthology.orchestra.run.smithi092.stdout:533.out 2023-06-09T04:37:31.907 INFO:teuthology.orchestra.run.smithi092.stdout:534 2023-06-09T04:37:31.907 INFO:teuthology.orchestra.run.smithi092.stdout:534.out 2023-06-09T04:37:31.907 INFO:teuthology.orchestra.run.smithi092.stdout:535 2023-06-09T04:37:31.908 INFO:teuthology.orchestra.run.smithi092.stdout:535.out 2023-06-09T04:37:31.908 INFO:teuthology.orchestra.run.smithi092.stdout:536 2023-06-09T04:37:31.908 INFO:teuthology.orchestra.run.smithi092.stdout:536.out 2023-06-09T04:37:31.908 INFO:teuthology.orchestra.run.smithi092.stdout:537 2023-06-09T04:37:31.909 INFO:teuthology.orchestra.run.smithi092.stdout:537.out 2023-06-09T04:37:31.909 INFO:teuthology.orchestra.run.smithi092.stdout:538 2023-06-09T04:37:31.909 INFO:teuthology.orchestra.run.smithi092.stdout:538.out 2023-06-09T04:37:31.909 INFO:teuthology.orchestra.run.smithi092.stdout:539 2023-06-09T04:37:31.910 INFO:teuthology.orchestra.run.smithi092.stdout:539.out 2023-06-09T04:37:31.910 INFO:teuthology.orchestra.run.smithi092.stdout:540 2023-06-09T04:37:31.910 INFO:teuthology.orchestra.run.smithi092.stdout:540.out 2023-06-09T04:37:31.910 INFO:teuthology.orchestra.run.smithi092.stdout:541 2023-06-09T04:37:31.911 INFO:teuthology.orchestra.run.smithi092.stdout:541.out 2023-06-09T04:37:31.911 INFO:teuthology.orchestra.run.smithi092.stdout:542 2023-06-09T04:37:31.911 INFO:teuthology.orchestra.run.smithi092.stdout:542.out 2023-06-09T04:37:31.911 INFO:teuthology.orchestra.run.smithi092.stdout:543 2023-06-09T04:37:31.912 INFO:teuthology.orchestra.run.smithi092.stdout:543.out 2023-06-09T04:37:31.912 INFO:teuthology.orchestra.run.smithi092.stdout:544 2023-06-09T04:37:31.912 INFO:teuthology.orchestra.run.smithi092.stdout:544.out 2023-06-09T04:37:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:545 2023-06-09T04:37:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:545.out 2023-06-09T04:37:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:546 2023-06-09T04:37:31.913 INFO:teuthology.orchestra.run.smithi092.stdout:546.out 2023-06-09T04:37:31.914 INFO:teuthology.orchestra.run.smithi092.stdout:547 2023-06-09T04:37:31.914 INFO:teuthology.orchestra.run.smithi092.stdout:547.out 2023-06-09T04:37:31.914 INFO:teuthology.orchestra.run.smithi092.stdout:548 2023-06-09T04:37:31.914 INFO:teuthology.orchestra.run.smithi092.stdout:548.out 2023-06-09T04:37:31.915 INFO:teuthology.orchestra.run.smithi092.stdout:549 2023-06-09T04:37:31.915 INFO:teuthology.orchestra.run.smithi092.stdout:549.out 2023-06-09T04:37:31.915 INFO:teuthology.orchestra.run.smithi092.stdout:550 2023-06-09T04:37:31.915 INFO:teuthology.orchestra.run.smithi092.stdout:550.out 2023-06-09T04:37:31.916 INFO:teuthology.orchestra.run.smithi092.stdout:551 2023-06-09T04:37:31.916 INFO:teuthology.orchestra.run.smithi092.stdout:551.out 2023-06-09T04:37:31.916 INFO:teuthology.orchestra.run.smithi092.stdout:552 2023-06-09T04:37:31.916 INFO:teuthology.orchestra.run.smithi092.stdout:552.out 2023-06-09T04:37:31.917 INFO:teuthology.orchestra.run.smithi092.stdout:553 2023-06-09T04:37:31.917 INFO:teuthology.orchestra.run.smithi092.stdout:553.out 2023-06-09T04:37:31.917 INFO:teuthology.orchestra.run.smithi092.stdout:554 2023-06-09T04:37:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:554.out 2023-06-09T04:37:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:555 2023-06-09T04:37:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:555.out 2023-06-09T04:37:31.918 INFO:teuthology.orchestra.run.smithi092.stdout:556 2023-06-09T04:37:31.919 INFO:teuthology.orchestra.run.smithi092.stdout:556.out 2023-06-09T04:37:31.919 INFO:teuthology.orchestra.run.smithi092.stdout:557 2023-06-09T04:37:31.919 INFO:teuthology.orchestra.run.smithi092.stdout:557.out 2023-06-09T04:37:31.919 INFO:teuthology.orchestra.run.smithi092.stdout:558 2023-06-09T04:37:31.920 INFO:teuthology.orchestra.run.smithi092.stdout:558.out 2023-06-09T04:37:31.920 INFO:teuthology.orchestra.run.smithi092.stdout:559 2023-06-09T04:37:31.920 INFO:teuthology.orchestra.run.smithi092.stdout:559.out 2023-06-09T04:37:31.920 INFO:teuthology.orchestra.run.smithi092.stdout:560 2023-06-09T04:37:31.921 INFO:teuthology.orchestra.run.smithi092.stdout:560.out 2023-06-09T04:37:31.921 INFO:teuthology.orchestra.run.smithi092.stdout:561 2023-06-09T04:37:31.921 INFO:teuthology.orchestra.run.smithi092.stdout:561.out 2023-06-09T04:37:31.921 INFO:teuthology.orchestra.run.smithi092.stdout:562 2023-06-09T04:37:31.922 INFO:teuthology.orchestra.run.smithi092.stdout:562.out 2023-06-09T04:37:31.922 INFO:teuthology.orchestra.run.smithi092.stdout:563 2023-06-09T04:37:31.922 INFO:teuthology.orchestra.run.smithi092.stdout:563.out 2023-06-09T04:37:31.922 INFO:teuthology.orchestra.run.smithi092.stdout:564 2023-06-09T04:37:31.923 INFO:teuthology.orchestra.run.smithi092.stdout:564.out 2023-06-09T04:37:31.923 INFO:teuthology.orchestra.run.smithi092.stdout:565 2023-06-09T04:37:31.923 INFO:teuthology.orchestra.run.smithi092.stdout:565.out 2023-06-09T04:37:31.923 INFO:teuthology.orchestra.run.smithi092.stdout:566 2023-06-09T04:37:31.924 INFO:teuthology.orchestra.run.smithi092.stdout:566.out 2023-06-09T04:37:31.924 INFO:teuthology.orchestra.run.smithi092.stdout:567 2023-06-09T04:37:31.924 INFO:teuthology.orchestra.run.smithi092.stdout:567.out 2023-06-09T04:37:31.925 INFO:teuthology.orchestra.run.smithi092.stdout:568 2023-06-09T04:37:31.925 INFO:teuthology.orchestra.run.smithi092.stdout:568.out 2023-06-09T04:37:31.925 INFO:teuthology.orchestra.run.smithi092.stdout:569 2023-06-09T04:37:31.925 INFO:teuthology.orchestra.run.smithi092.stdout:569.out 2023-06-09T04:37:31.926 INFO:teuthology.orchestra.run.smithi092.stdout:570 2023-06-09T04:37:31.926 INFO:teuthology.orchestra.run.smithi092.stdout:570.out 2023-06-09T04:37:31.926 INFO:teuthology.orchestra.run.smithi092.stdout:571 2023-06-09T04:37:31.926 INFO:teuthology.orchestra.run.smithi092.stdout:571.out 2023-06-09T04:37:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:572 2023-06-09T04:37:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:572.out 2023-06-09T04:37:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:573 2023-06-09T04:37:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:573.out 2023-06-09T04:37:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:574 2023-06-09T04:37:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:574.out 2023-06-09T04:37:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:575 2023-06-09T04:37:31.928 INFO:teuthology.orchestra.run.smithi092.stdout:575.out 2023-06-09T04:37:31.928 INFO:teuthology.orchestra.run.smithi092.stdout:576 2023-06-09T04:37:31.928 INFO:teuthology.orchestra.run.smithi092.stdout:576.out 2023-06-09T04:37:31.928 INFO:teuthology.orchestra.run.smithi092.stdout:577 2023-06-09T04:37:31.928 INFO:teuthology.orchestra.run.smithi092.stdout:577.out 2023-06-09T04:37:31.928 INFO:teuthology.orchestra.run.smithi092.stdout:578 2023-06-09T04:37:31.928 INFO:teuthology.orchestra.run.smithi092.stdout:578.out 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:579 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:579.out 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:580 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:580.out 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:581 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:581.out 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:582 2023-06-09T04:37:31.929 INFO:teuthology.orchestra.run.smithi092.stdout:582.out 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:583 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:583.out 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:584 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:584.out 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:585 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:585.out 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:586 2023-06-09T04:37:31.930 INFO:teuthology.orchestra.run.smithi092.stdout:586.out 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:587 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:587.out 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:588 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:588.out 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:589 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:589.out 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:590 2023-06-09T04:37:31.931 INFO:teuthology.orchestra.run.smithi092.stdout:590.out 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:591 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:591.out 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:592 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:592.out 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:593 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:593.out 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:594 2023-06-09T04:37:31.932 INFO:teuthology.orchestra.run.smithi092.stdout:594.out 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:595 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:595.out 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:596 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:596.out 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:597 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:597.out 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:598 2023-06-09T04:37:31.933 INFO:teuthology.orchestra.run.smithi092.stdout:598.out 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:599 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:599.out 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:600 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:600.out 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:601 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:601.out 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:602 2023-06-09T04:37:31.934 INFO:teuthology.orchestra.run.smithi092.stdout:602.out 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:603 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:603.out 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:604 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:604.out 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:605 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:605.out 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:606 2023-06-09T04:37:31.935 INFO:teuthology.orchestra.run.smithi092.stdout:606.out 2023-06-09T04:37:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:607 2023-06-09T04:37:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:607.out 2023-06-09T04:37:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:608 2023-06-09T04:37:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:608.out 2023-06-09T04:37:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:609 2023-06-09T04:37:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:609.out 2023-06-09T04:37:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:610 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:610.out 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:611 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:611.out 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:612 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:612.out 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:613 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:613.out 2023-06-09T04:37:31.937 INFO:teuthology.orchestra.run.smithi092.stdout:614 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:614.out 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:615 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:615.out 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:616 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:616.out 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:617 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:617.out 2023-06-09T04:37:31.938 INFO:teuthology.orchestra.run.smithi092.stdout:618 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:618.out 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:619 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:619.out 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:620 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:620.out 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:621 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:621.out 2023-06-09T04:37:31.939 INFO:teuthology.orchestra.run.smithi092.stdout:622 2023-06-09T04:37:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:622.out 2023-06-09T04:37:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:623 2023-06-09T04:37:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:623.out 2023-06-09T04:37:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:624 2023-06-09T04:37:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:624.out 2023-06-09T04:37:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:625 2023-06-09T04:37:31.940 INFO:teuthology.orchestra.run.smithi092.stdout:625.out 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:626 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:626.out 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:627 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:627.out 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:628 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:628.out 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:629 2023-06-09T04:37:31.941 INFO:teuthology.orchestra.run.smithi092.stdout:629.out 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:630 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:630.out 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:631 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:631.out 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:632 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:632.out 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:633 2023-06-09T04:37:31.942 INFO:teuthology.orchestra.run.smithi092.stdout:633.out 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:634 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:634.out 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:635 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:635.out 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:636 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:636.out 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:637 2023-06-09T04:37:31.943 INFO:teuthology.orchestra.run.smithi092.stdout:637.out 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:638 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:638.out 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:639 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:639.out 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:640 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:640.out 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:641 2023-06-09T04:37:31.944 INFO:teuthology.orchestra.run.smithi092.stdout:641.out 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:642 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:642.out 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:643 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:643.out 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:644 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:644.out 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:645 2023-06-09T04:37:31.945 INFO:teuthology.orchestra.run.smithi092.stdout:645.out 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:646 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:646.out 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:647 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:647.out 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:648 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:648.out 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:649 2023-06-09T04:37:31.946 INFO:teuthology.orchestra.run.smithi092.stdout:649.out 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:650 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:650.out 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:651 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:651.out 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:652 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:652.out 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:653 2023-06-09T04:37:31.947 INFO:teuthology.orchestra.run.smithi092.stdout:653.out 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:654 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:654.out 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:655 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:655.out 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:656 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:656.out 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:657 2023-06-09T04:37:31.948 INFO:teuthology.orchestra.run.smithi092.stdout:657.out 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:658 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:658.out 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:659 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:659.out 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:660 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:660.out 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:661 2023-06-09T04:37:31.949 INFO:teuthology.orchestra.run.smithi092.stdout:661.out 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:662 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:662.out 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:663 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:663.out 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:664 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:664.out 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:665 2023-06-09T04:37:31.950 INFO:teuthology.orchestra.run.smithi092.stdout:665.out 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:666 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:666.out 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:667 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:667.out 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:668 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:668.out 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:669 2023-06-09T04:37:31.951 INFO:teuthology.orchestra.run.smithi092.stdout:669.out 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:670 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:670.out 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:671 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:671.out 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:672 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:672.out 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:673 2023-06-09T04:37:31.952 INFO:teuthology.orchestra.run.smithi092.stdout:673.out 2023-06-09T04:37:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:674 2023-06-09T04:37:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:674.out 2023-06-09T04:37:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:675 2023-06-09T04:37:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:675.out 2023-06-09T04:37:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:676 2023-06-09T04:37:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:676.out 2023-06-09T04:37:31.953 INFO:teuthology.orchestra.run.smithi092.stdout:677 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:677.out 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:678 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:678.out 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:679 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:679.out 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:680 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:680.out 2023-06-09T04:37:31.954 INFO:teuthology.orchestra.run.smithi092.stdout:681 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:681.out 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:682 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:682.out 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:683 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:683.out 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:684 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:684.out 2023-06-09T04:37:31.955 INFO:teuthology.orchestra.run.smithi092.stdout:685 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:685.out 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:686 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:686.out 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:687 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:687.out 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:688 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:688.out 2023-06-09T04:37:31.956 INFO:teuthology.orchestra.run.smithi092.stdout:689 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:689.out 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:690 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:690.out 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:691 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:691.out 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:692 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:692.out 2023-06-09T04:37:31.957 INFO:teuthology.orchestra.run.smithi092.stdout:693 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:693.out 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:694 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:694.out 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:695 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:695.out 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:696 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:696.out 2023-06-09T04:37:31.958 INFO:teuthology.orchestra.run.smithi092.stdout:697 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:697.out 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:698 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:698.out 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:699 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:699.out 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:700 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:700.out 2023-06-09T04:37:31.959 INFO:teuthology.orchestra.run.smithi092.stdout:701 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:701.out 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:702 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:702.out 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:703 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:703.out 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:704 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:704.out 2023-06-09T04:37:31.960 INFO:teuthology.orchestra.run.smithi092.stdout:705 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:705.out 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:706 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:706.out 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:707 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:707.out 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:708 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:708.out 2023-06-09T04:37:31.961 INFO:teuthology.orchestra.run.smithi092.stdout:709 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:709.out 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:710 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:710.out 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:711 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:711.out 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:712 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:712.out 2023-06-09T04:37:31.962 INFO:teuthology.orchestra.run.smithi092.stdout:713 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:713.out 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:714 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:714.out 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:715 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:715.out 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:716 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:716.out 2023-06-09T04:37:31.963 INFO:teuthology.orchestra.run.smithi092.stdout:717 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:717.out 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:718 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:718.out 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:719 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:719.out 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:720 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:720.out 2023-06-09T04:37:31.964 INFO:teuthology.orchestra.run.smithi092.stdout:721 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:721.out 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:722 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:722.out 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:723 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:723.out 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:724 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:724.out 2023-06-09T04:37:31.965 INFO:teuthology.orchestra.run.smithi092.stdout:725 2023-06-09T04:37:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:725.out 2023-06-09T04:37:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:726 2023-06-09T04:37:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:726.out 2023-06-09T04:37:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:727 2023-06-09T04:37:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:727.out 2023-06-09T04:37:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:group.list 2023-06-09T04:37:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:Makefile 2023-06-09T04:37:31.967 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-06-09T04:37:32.184 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:37:32.188 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:37:32.189 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:37:32.190 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:37:32.190 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:36.957 INFO:teuthology.orchestra.run.smithi092.stdout:generic/001 4s 2023-06-09T04:37:36.962 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/001 2023-06-09T04:37:36.962 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:37:36.964 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:37.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:37:37.670 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-06-09T04:37:37.874 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:37:37.878 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:37:37.879 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:37:37.880 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:37:37.880 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:38.480 INFO:teuthology.orchestra.run.smithi092.stdout:generic/002 0s 2023-06-09T04:37:38.485 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/002 2023-06-09T04:37:38.485 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:37:38.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:38.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:37:38.552 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-06-09T04:37:38.760 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:37:38.764 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:37:38.764 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:37:38.765 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:37:38.765 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:39.079 INFO:teuthology.orchestra.run.smithi092.stdout:generic/003 [not run] atime not maintained by ceph 2023-06-09T04:37:39.082 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/003 2023-06-09T04:37:39.083 INFO:teuthology.orchestra.run.smithi092.stdout:Not run: generic/003 2023-06-09T04:37:39.083 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:37:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:39.104 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:37:39.105 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-06-09T04:37:39.313 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:37:39.317 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:37:39.317 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:37:39.318 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:37:39.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:39.642 INFO:teuthology.orchestra.run.smithi092.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-06-09T04:37:39.645 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/004 2023-06-09T04:37:39.646 INFO:teuthology.orchestra.run.smithi092.stdout:Not run: generic/004 2023-06-09T04:37:39.646 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:37:39.648 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:39.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:37:39.675 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-06-09T04:37:39.896 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:37:39.900 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:37:39.900 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:37:39.901 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:37:39.901 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:40.406 INFO:teuthology.orchestra.run.smithi092.stdout:generic/005 0s 2023-06-09T04:37:40.451 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/005 2023-06-09T04:37:40.452 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:37:40.453 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:40.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:37:40.503 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-06-09T04:37:40.709 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:37:40.713 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:37:40.714 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:37:40.715 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:37:40.715 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:37:46.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:37:46.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:38:05.069 INFO:teuthology.orchestra.run.smithi092.stdout:generic/006 25s 2023-06-09T04:38:05.119 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/006 2023-06-09T04:38:05.119 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:38:05.121 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:05.178 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:38:05.179 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-06-09T04:38:05.562 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:38:05.565 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:38:05.566 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:38:05.567 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:38:05.567 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:16.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:38:16.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:38:46.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:38:46.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:38:55.754 INFO:teuthology.orchestra.run.smithi092.stdout:generic/007 50s 2023-06-09T04:38:55.820 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/007 2023-06-09T04:38:55.820 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:38:55.822 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:55.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:38:55.894 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-06-09T04:38:56.232 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:38:56.236 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:38:56.236 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:38:56.237 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:38:56.238 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:56.605 INFO:teuthology.orchestra.run.smithi092.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-06-09T04:38:56.608 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/008 2023-06-09T04:38:56.609 INFO:teuthology.orchestra.run.smithi092.stdout:Not run: generic/008 2023-06-09T04:38:56.609 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:38:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:56.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:38:56.634 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-06-09T04:38:56.847 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:38:56.851 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:38:56.852 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:38:56.852 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:38:56.853 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:57.224 INFO:teuthology.orchestra.run.smithi092.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-06-09T04:38:57.228 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/009 2023-06-09T04:38:57.228 INFO:teuthology.orchestra.run.smithi092.stdout:Not run: generic/009 2023-06-09T04:38:57.228 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:38:57.230 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:57.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:38:57.257 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-06-09T04:38:57.456 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:38:57.460 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:38:57.461 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:38:57.461 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:38:57.462 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:57.758 INFO:teuthology.orchestra.run.smithi092.stdout:generic/010 [not run] /tmp/tmp.3HuE1HNmdOxfstests-dev/src/dbtest not built 2023-06-09T04:38:57.762 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/010 2023-06-09T04:38:57.762 INFO:teuthology.orchestra.run.smithi092.stdout:Not run: generic/010 2023-06-09T04:38:57.763 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:38:57.764 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:38:57.784 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:38:57.785 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-06-09T04:38:57.979 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:38:57.983 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:38:57.984 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:38:57.984 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:38:57.985 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:39:16.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:39:16.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:39:30.884 INFO:teuthology.orchestra.run.smithi092.stdout:generic/011 32s 2023-06-09T04:39:30.893 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/011 2023-06-09T04:39:30.893 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:39:30.895 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:39:31.194 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:39:31.195 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-06-09T04:39:31.503 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:39:31.507 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:39:31.508 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:39:31.508 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:39:31.509 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:39:31.905 INFO:teuthology.orchestra.run.smithi092.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2023-06-09T04:39:31.908 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/012 2023-06-09T04:39:31.909 INFO:teuthology.orchestra.run.smithi092.stdout:Not run: generic/012 2023-06-09T04:39:31.909 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:39:31.910 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:39:31.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:39:31.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-06-09T04:39:32.198 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:39:32.202 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:39:32.202 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:39:32.203 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:39:32.203 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:39:46.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:39:46.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:40:16.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:40:16.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:40:17.304 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f56ff7f3700 time 2023-06-09T04:40:17.304374+0000 2023-06-09T04:40:17.304 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-09T04:40:17.305 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.305 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f57110b7dbf] 2023-06-09T04:40:17.306 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.306 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.306 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.306 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.306 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.306 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.306 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: clone() 2023-06-09T04:40:17.307 INFO:tasks.ceph.mds.a.smithi092.stderr:*** Caught signal (Aborted) ** 2023-06-09T04:40:17.308 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56ff7f3700 thread_name:MR_Finisher 2023-06-09T04:40:17.308 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-06-09T04:40:17.304+0000 7f56ff7f3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f56ff7f3700 time 2023-06-09T04:40:17.304374+0000 2023-06-09T04:40:17.308 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f57110b7dbf] 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.309 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.310 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.310 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.310 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.310 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.310 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.310 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.310 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: clone() 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f570fe6ace0] 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: gsignal() 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: abort() 2023-06-09T04:40:17.311 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f57110b7e19] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.312 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.313 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.313 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.313 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.313 INFO:tasks.ceph.mds.a.smithi092.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.313 INFO:tasks.ceph.mds.a.smithi092.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.313 INFO:tasks.ceph.mds.a.smithi092.stderr: 18: clone() 2023-06-09T04:40:17.313 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-06-09T04:40:17.306+0000 7f56ff7f3700 -1 *** Caught signal (Aborted) ** 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56ff7f3700 thread_name:MR_Finisher 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f570fe6ace0] 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: gsignal() 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: abort() 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f57110b7e19] 2023-06-09T04:40:17.314 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.315 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.316 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.316 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.316 INFO:tasks.ceph.mds.a.smithi092.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.316 INFO:tasks.ceph.mds.a.smithi092.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.316 INFO:tasks.ceph.mds.a.smithi092.stderr: 18: clone() 2023-06-09T04:40:17.316 INFO:tasks.ceph.mds.a.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-09T04:40:17.316 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.328 INFO:tasks.ceph.mds.a.smithi092.stderr: -2> 2023-06-09T04:40:17.304+0000 7f56ff7f3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f56ff7f3700 time 2023-06-09T04:40:17.304374+0000 2023-06-09T04:40:17.328 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-09T04:40:17.328 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.328 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.328 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f57110b7dbf] 2023-06-09T04:40:17.328 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.328 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.329 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.329 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.329 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.329 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.329 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.329 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.329 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: clone() 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: 0> 2023-06-09T04:40:17.306+0000 7f56ff7f3700 -1 *** Caught signal (Aborted) ** 2023-06-09T04:40:17.330 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56ff7f3700 thread_name:MR_Finisher 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f570fe6ace0] 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: gsignal() 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: abort() 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f57110b7e19] 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.331 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.332 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.333 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.333 INFO:tasks.ceph.mds.a.smithi092.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.333 INFO:tasks.ceph.mds.a.smithi092.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.333 INFO:tasks.ceph.mds.a.smithi092.stderr: 18: clone() 2023-06-09T04:40:17.333 INFO:tasks.ceph.mds.a.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-09T04:40:17.333 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.344 INFO:tasks.ceph.mds.a.smithi092.stderr: -82> 2023-06-09T04:40:17.304+0000 7f56ff7f3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f56ff7f3700 time 2023-06-09T04:40:17.304374+0000 2023-06-09T04:40:17.345 INFO:tasks.ceph.mds.a.smithi092.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-09T04:40:17.345 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.345 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.345 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f57110b7dbf] 2023-06-09T04:40:17.345 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.345 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.346 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: clone() 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: -80> 2023-06-09T04:40:17.306+0000 7f56ff7f3700 -1 *** Caught signal (Aborted) ** 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: in thread 7f56ff7f3700 thread_name:MR_Finisher 2023-06-09T04:40:17.347 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f570fe6ace0] 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: 2: gsignal() 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: 3: abort() 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f57110b7e19] 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f57110b7f85] 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x55790980d432] 2023-06-09T04:40:17.348 INFO:tasks.ceph.mds.a.smithi092.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x5579097f534c] 2023-06-09T04:40:17.349 INFO:tasks.ceph.mds.a.smithi092.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x5579097fe187] 2023-06-09T04:40:17.349 INFO:tasks.ceph.mds.a.smithi092.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x5579098029ff] 2023-06-09T04:40:17.349 INFO:tasks.ceph.mds.a.smithi092.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x557909654bd7] 2023-06-09T04:40:17.349 INFO:tasks.ceph.mds.a.smithi092.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x557909655a80] 2023-06-09T04:40:17.349 INFO:tasks.ceph.mds.a.smithi092.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x557909673c3a] 2023-06-09T04:40:17.349 INFO:tasks.ceph.mds.a.smithi092.stderr: 13: (MDSContext::complete(int)+0x5f) [0x55790993c18f] 2023-06-09T04:40:17.349 INFO:tasks.ceph.mds.a.smithi092.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x55790993c914] 2023-06-09T04:40:17.350 INFO:tasks.ceph.mds.a.smithi092.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x55790993cccb] 2023-06-09T04:40:17.350 INFO:tasks.ceph.mds.a.smithi092.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f571115b7cd] 2023-06-09T04:40:17.350 INFO:tasks.ceph.mds.a.smithi092.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f570fe601cf] 2023-06-09T04:40:17.350 INFO:tasks.ceph.mds.a.smithi092.stderr: 18: clone() 2023-06-09T04:40:17.350 INFO:tasks.ceph.mds.a.smithi092.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-09T04:40:17.350 INFO:tasks.ceph.mds.a.smithi092.stderr: 2023-06-09T04:40:18.549 INFO:tasks.ceph.mds.a.smithi092.stderr:daemon-helper: command crashed with signal 6 2023-06-09T04:40:25.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-06-09T04:40:32.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-06-09T04:40:38.628 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-06-09T04:40:45.237 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2023-06-09T04:40:46.286 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-06-09T04:40:46.286+0000 7f186fdb7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T04:40:46.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:40:46.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:40:51.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~26s 2023-06-09T04:40:58.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~33s 2023-06-09T04:41:05.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~40s 2023-06-09T04:41:11.671 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~46s 2023-06-09T04:41:13.262 INFO:teuthology.orchestra.run.smithi092.stdout:generic/013 101s 2023-06-09T04:41:13.273 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/013 2023-06-09T04:41:13.273 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:41:13.279 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:41:13.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:41:13.325 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-06-09T04:41:13.756 INFO:teuthology.orchestra.run.smithi092.stdout:FSTYP -- ceph 2023-06-09T04:41:13.765 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM -- Linux/x86_64 smithi092 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-09T04:41:13.765 INFO:teuthology.orchestra.run.smithi092.stdout:MKFS_OPTIONS -- 172.21.15.92:6789:/scratch 2023-06-09T04:41:13.767 INFO:teuthology.orchestra.run.smithi092.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQDJq4Jk1PBrORAAdpg4Ffw/5rlBjDy0uZCQWA== -o context=system_u:object_r:root_t:s0 172.21.15.92:6789:/scratch /tmp/tmp.avGnPmg37Fscratch 2023-06-09T04:41:13.767 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:41:16.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:41:16.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:41:18.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~53s 2023-06-09T04:41:24.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~59s 2023-06-09T04:41:31.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~66s 2023-06-09T04:41:38.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~73s 2023-06-09T04:41:44.717 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~79s 2023-06-09T04:41:47.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:41:47.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:41:51.328 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~86s 2023-06-09T04:41:57.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~93s 2023-06-09T04:42:04.550 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~99s 2023-06-09T04:42:11.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~106s 2023-06-09T04:42:17.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:42:17.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:42:17.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~112s 2023-06-09T04:42:24.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~119s 2023-06-09T04:42:30.992 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~126s 2023-06-09T04:42:37.603 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~132s 2023-06-09T04:42:44.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~139s 2023-06-09T04:42:48.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:42:48.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:42:50.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~145s 2023-06-09T04:42:57.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~152s 2023-06-09T04:43:04.042 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~159s 2023-06-09T04:43:10.650 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~165s 2023-06-09T04:43:17.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~172s 2023-06-09T04:43:18.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:43:18.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:43:23.869 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~178s 2023-06-09T04:43:30.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~185s 2023-06-09T04:43:37.088 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~192s 2023-06-09T04:43:43.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~198s 2023-06-09T04:43:48.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:43:48.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:43:50.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~205s 2023-06-09T04:43:56.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~212s 2023-06-09T04:44:03.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~218s 2023-06-09T04:44:10.139 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~225s 2023-06-09T04:44:16.749 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~231s 2023-06-09T04:44:19.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:44:19.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:44:23.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~238s 2023-06-09T04:44:29.970 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~245s 2023-06-09T04:44:36.580 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~251s 2023-06-09T04:44:43.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~258s 2023-06-09T04:44:49.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:44:49.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:44:49.801 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~264s 2023-06-09T04:44:56.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~271s 2023-06-09T04:45:03.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~278s 2023-06-09T04:45:09.633 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~284s 2023-06-09T04:45:16.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~291s 2023-06-09T04:45:19.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:45:19.901 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:45:22.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~297s 2023-06-09T04:45:29.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~304s 2023-06-09T04:45:29.466 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-06-09T04:45:29.466 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:45:29.467 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:45:29.517 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:45:29.518 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:45:29.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T04:45:29.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.818+0000 7f5006497700 1 Processor -- start 2023-06-09T04:45:29.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.818+0000 7f5006497700 1 -- start start 2023-06-09T04:45:29.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f5006497700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50001586f0 0x7f5000070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:29.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f5006497700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5000071210 con 0x7f50000c92d0 2023-06-09T04:45:29.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f5006497700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500006aa10 con 0x7f50000c8e80 2023-06-09T04:45:29.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f5006497700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50000ae180 con 0x7f50001586f0 2023-06-09T04:45:29.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f4ffffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50001586f0 0x7f5000070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:29.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f4ffffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f50001586f0 0x7f5000070cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40106/0 (socket says 172.21.15.92:40106) 2023-06-09T04:45:29.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f4ffffff700 1 -- 172.21.15.92:0/2610640841 learned_addr learned my addr 172.21.15.92:0/2610640841 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:29.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.819+0000 7f5004a34700 1 --1- 172.21.15.92:0/2610640841 >> v1:172.21.15.92:6789/0 conn(0x7f50000c92d0 0x7f50000cc590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38440/0 (socket says 172.21.15.92:38440) 2023-06-09T04:45:29.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.820+0000 7f4ffffff700 1 -- 172.21.15.92:0/2610640841 >> v1:172.21.15.123:6789/0 conn(0x7f50000c8e80 legacy=0x7f500015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:45:29.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.820+0000 7f4ffffff700 1 -- 172.21.15.92:0/2610640841 >> v1:172.21.15.92:6789/0 conn(0x7f50000c92d0 legacy=0x7f50000cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:29.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.820+0000 7f4ffffff700 1 -- 172.21.15.92:0/2610640841 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50000c9750 con 0x7f50001586f0 2023-06-09T04:45:29.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.820+0000 7f4ffffff700 1 --2- 172.21.15.92:0/2610640841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50001586f0 0x7f5000070cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00088f0 tx=0x7f4ff000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd51d2f3589d0865 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:29.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.820+0000 7f4ffeffd700 1 -- 172.21.15.92:0/2610640841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0010070 con 0x7f50001586f0 2023-06-09T04:45:29.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.821+0000 7f4ffeffd700 1 -- 172.21.15.92:0/2610640841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff000da40 con 0x7f50001586f0 2023-06-09T04:45:29.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.821+0000 7f4ffeffd700 1 -- 172.21.15.92:0/2610640841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000ddb0 con 0x7f50001586f0 2023-06-09T04:45:29.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.821+0000 7f5006497700 1 -- 172.21.15.92:0/2610640841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50001586f0 msgr2=0x7f5000070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:29.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.821+0000 7f5006497700 1 --2- 172.21.15.92:0/2610640841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50001586f0 0x7f5000070cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00088f0 tx=0x7f4ff000d330 comp rx=0 tx=0).stop 2023-06-09T04:45:29.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.825+0000 7f5006497700 1 -- 172.21.15.92:0/2610640841 shutdown_connections 2023-06-09T04:45:29.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.825+0000 7f5006497700 1 --2- 172.21.15.92:0/2610640841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50001586f0 0x7f5000070cd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:29.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.825+0000 7f5006497700 1 -- 172.21.15.92:0/2610640841 >> 172.21.15.92:0/2610640841 conn(0x7f50000b0ff0 msgr2=0x7f50000ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:29.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.825+0000 7f5006497700 1 -- 172.21.15.92:0/2610640841 shutdown_connections 2023-06-09T04:45:29.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.825+0000 7f5006497700 1 -- 172.21.15.92:0/2610640841 wait complete. 2023-06-09T04:45:29.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.827+0000 7f5006497700 1 Processor -- start 2023-06-09T04:45:29.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.827+0000 7f5006497700 1 -- start start 2023-06-09T04:45:29.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f5006497700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50000c8e80 0x7f50000c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:29.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f5006497700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50000c92d0 0x7f50000b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:29.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f5006497700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50001586f0 0x7f500014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:29.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f5006497700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000b3710 con 0x7f50000c8e80 2023-06-09T04:45:29.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f5006497700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000b28a0 con 0x7f50001586f0 2023-06-09T04:45:29.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f5006497700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5000150300 con 0x7f50000c92d0 2023-06-09T04:45:29.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f4ffffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50000c8e80 0x7f50000c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:29.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f4ffffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50000c8e80 0x7f50000c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40110/0 (socket says 172.21.15.92:40110) 2023-06-09T04:45:29.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.828+0000 7f4ffffff700 1 -- 172.21.15.92:0/1047239657 learned_addr learned my addr 172.21.15.92:0/1047239657 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:29.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffffff700 1 -- 172.21.15.92:0/1047239657 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50000c92d0 msgr2=0x7f50000b2360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T04:45:29.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffffff700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50000c92d0 0x7f50000b2360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffffff700 1 -- 172.21.15.92:0/1047239657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50001586f0 msgr2=0x7f500014fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffffff700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50001586f0 0x7f500014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffffff700 1 -- 172.21.15.92:0/1047239657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff00085d0 con 0x7f50000c8e80 2023-06-09T04:45:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffffff700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50000c8e80 0x7f50000c49f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000d370 tx=0x7f4ff0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffcff9700 1 -- 172.21.15.92:0/1047239657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0008d90 con 0x7f50000c8e80 2023-06-09T04:45:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50000b2b00 con 0x7f50000c8e80 2023-06-09T04:45:29.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50000b30c0 con 0x7f50000c8e80 2023-06-09T04:45:29.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffcff9700 1 -- 172.21.15.92:0/1047239657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff00038d0 con 0x7f50000c8e80 2023-06-09T04:45:29.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.829+0000 7f4ffcff9700 1 -- 172.21.15.92:0/1047239657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0003c80 con 0x7f50000c8e80 2023-06-09T04:45:29.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.831+0000 7f4ffcff9700 1 -- 172.21.15.92:0/1047239657 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ff000b160 con 0x7f50000c8e80 2023-06-09T04:45:29.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.832+0000 7f4ffcff9700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4fe406acb0 0x7f4fe406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:29.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.832+0000 7f4ffcff9700 1 -- 172.21.15.92:0/1047239657 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f4ff008a010 con 0x7f50000c8e80 2023-06-09T04:45:29.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.833+0000 7f4fff7fe700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4fe406acb0 0x7f4fe406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:29.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.834+0000 7f4fff7fe700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4fe406acb0 0x7f4fe406d160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f500006d4a0 tx=0x7f4ff4007a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:29.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.837+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc0052f0 con 0x7f50000c8e80 2023-06-09T04:45:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:29.843+0000 7f4ffcff9700 1 -- 172.21.15.92:0/1047239657 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ff0055050 con 0x7f50000c8e80 2023-06-09T04:45:30.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.193+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fdc005160 con 0x7f50000c8e80 2023-06-09T04:45:30.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.194+0000 7f4ffcff9700 1 -- 172.21.15.92:0/1047239657 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18447 (secure 0 0 0) 0x7f4ff0024090 con 0x7f50000c8e80 2023-06-09T04:45:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:45:30.196 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"fsid":"064bde79-2943-47e5-b941-05f234bd883f","created":"2023-06-09T04:34:52.296241+0000","modified":"2023-06-09T04:40:32.122220+0000","last_up_change":"2023-06-09T04:35:03.903067+0000","last_in_change":"2023-06-09T04:34:54.721798+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T04:35:05.886152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T04:35:10.674657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-09T04:36:11.118404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-09T04:36:11.791966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c9fd229-6037-49a3-a19d-2e9b9f077b05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6809","nonce":3209682425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6811","nonce":3209682425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6815","nonce":3209682425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":3209682425},{"type":"v1","addr":"172.21.15.92:6813","nonce":3209682425}]},"public_addr":"172.21.15.92:6809/3209682425","cluster_addr":"172.21.15.92:6811/3209682425","heartbeat_back_addr":"172.21.15.92:6815/3209682425","heartbeat_front_addr":"172.21.15.92:6813/3209682425","state":["exists","up"]},{"osd":1,"uuid":"9517b22e-f8d8-469d-9805-b7d3018f29bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6801","nonce":3648230153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6803","nonce":3648230153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6807","nonce":3648230153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3648230153},{"type":"v1","addr":"172.21.15.92:6805","nonce":3648230153}]},"public_addr":"172.21.15.92:6801/3648230153","cluster_addr":"172.21.15.92:6803/3648230153","heartbeat_back_addr":"172.21.15.92:6807/3648230153","heartbeat_front_addr":"172.21.15.92:6805/3648230153","state":["exists","up"]},{"osd":2,"uuid":"3aac59e9-dc25-4242-8210-8ea207bec83a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6825","nonce":445681078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6827","nonce":445681078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6831","nonce":445681078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":445681078},{"type":"v1","addr":"172.21.15.92:6829","nonce":445681078}]},"public_addr":"172.21.15.92:6825/445681078","cluster_addr":"172.21.15.92:6827/445681078","heartbeat_back_addr":"172.21.15.92:6831/445681078","heartbeat_front_addr":"172.21.15.92:6829/445681078","state":["exists","up"]},{"osd":3,"uuid":"e4e32b61-763b-465f-92e8-77267ad85fc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6817","nonce":2565093508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6819","nonce":2565093508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6823","nonce":2565093508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":2565093508},{"type":"v1","addr":"172.21.15.92:6821","nonce":2565093508}]},"public_addr":"172.21.15.92:6817/2565093508","cluster_addr":"172.21.15.92:6819/2565093508","heartbeat_back_addr":"172.21.15.92:6823/2565093508","heartbeat_front_addr":"172.21.15.92:6821/2565093508","state":["exists","up"]},{"osd":4,"uuid":"69d5fe7c-90b1-434f-afaa-90d341672a14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6801","nonce":909955472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6803","nonce":909955472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6807","nonce":909955472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":909955472},{"type":"v1","addr":"172.21.15.123:6805","nonce":909955472}]},"public_addr":"172.21.15.123:6801/909955472","cluster_addr":"172.21.15.123:6803/909955472","heartbeat_back_addr":"172.21.15.123:6807/909955472","heartbeat_front_addr":"172.21.15.123:6805/909955472","state":["exists","up"]},{"osd":5,"uuid":"ad7f134c-14ea-49aa-9bcb-b0c46898834d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6809","nonce":3070586225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6811","nonce":3070586225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6815","nonce":3070586225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3070586225},{"type":"v1","addr":"172.21.15.123:6813","nonce":3070586225}]},"public_addr":"172.21.15.123:6809/3070586225","cluster_addr":"172.21.15.123:6811/3070586225","heartbeat_back_addr":"172.21.15.123:6815/3070586225","heartbeat_front_addr":"172.21.15.123:6813/3070586225","state":["exists","up"]},{"osd":6,"uuid":"ec5c5ee3-4cc5-4751-8576-2e584f15095d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6825","nonce":528556293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6827","nonce":528556293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6831","nonce":528556293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":528556293},{"type":"v1","addr":"172.21.15.123:6829","nonce":528556293}]},"public_addr":"172.21.15.123:6825/528556293","cluster_addr":"172.21.15.123:6827/528556293","heartbeat_back_addr":"172.21.15.123:6831/528556293","heartbeat_front_addr":"172.21.15.123:6829/528556293","state":["exists","up"]},{"osd":7,"uuid":"2606c92d-19ca-4b65-b058-ea5e8de3c0a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6817","nonce":3620631491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6819","nonce":3620631491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6823","nonce":3620631491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":3620631491},{"type":"v1","addr":"172.21.15.123:6821","nonce":3620631491}]},"public_addr":"172.21.15.123:6817/3620631491","cluster_addr":"172.21.15.123:6819/3620631491","heartbeat_back_addr":"172.21.15.123:6823/3620631491","heartbeat_front_addr":"172.21.15.123:6821/3620631491","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.940471+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.921594+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.923382+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.917041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.944433+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:35:00.006665+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.967568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T04:34:59.985380+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.36","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6837/637518437":"2023-06-10T04:40:32.122135+0000","172.21.15.92:6835/637518437":"2023-06-10T04:40:32.122135+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T04:45:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.196+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4fe406acb0 msgr2=0x7f4fe406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.196+0000 7f5006497700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4fe406acb0 0x7f4fe406d160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f500006d4a0 tx=0x7f4ff4007a00 comp rx=0 tx=0).stop 2023-06-09T04:45:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.196+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50000c8e80 msgr2=0x7f50000c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.196+0000 7f5006497700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50000c8e80 0x7f50000c49f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000d370 tx=0x7f4ff0004750 comp rx=0 tx=0).stop 2023-06-09T04:45:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 shutdown_connections 2023-06-09T04:45:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f4fe406acb0 0x7f4fe406d160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50001586f0 0x7f500014fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50000c8e80 0x7f50000c49f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 --2- 172.21.15.92:0/1047239657 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50000c92d0 0x7f50000b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 >> 172.21.15.92:0/1047239657 conn(0x7f50000b0ff0 msgr2=0x7f5000152970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 shutdown_connections 2023-06-09T04:45:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.197+0000 7f5006497700 1 -- 172.21.15.92:0/1047239657 wait complete. 2023-06-09T04:45:30.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T04:35:05.886152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T04:35:10.674657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T04:36:11.118404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T04:36:11.791966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T04:45:30.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T04:45:30.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.418+0000 7f69f29cc700 1 Processor -- start 2023-06-09T04:45:30.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.419+0000 7f69f29cc700 1 -- start start 2023-06-09T04:45:30.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69f29cc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69ec15a0f0 0x7f69ec0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:30.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69f29cc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec15a4c0 con 0x7f69ec0cb740 2023-06-09T04:45:30.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69f29cc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec0cbb40 con 0x7f69ec0ce250 2023-06-09T04:45:30.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69f29cc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec070c40 con 0x7f69ec15a0f0 2023-06-09T04:45:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69ebfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69ec15a0f0 0x7f69ec0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69f0f69700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f69ec0cb740 0x7f69ec0cc620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38452/0 (socket says 172.21.15.92:38452) 2023-06-09T04:45:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69f0f69700 1 -- 172.21.15.92:0/2907085950 learned_addr learned my addr 172.21.15.92:0/2907085950 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:45:30.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69ebfff700 1 --2- 172.21.15.92:0/2907085950 >> v2:172.21.15.92:3300/0 conn(0x7f69ec15a0f0 0x7f69ec0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40118/0 (socket says 172.21.15.92:40118) 2023-06-09T04:45:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69ebfff700 1 -- 172.21.15.92:0/2907085950 >> v1:172.21.15.123:6789/0 conn(0x7f69ec0ce250 legacy=0x7f69ec0cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69ebfff700 1 -- 172.21.15.92:0/2907085950 >> v1:172.21.15.92:6789/0 conn(0x7f69ec0cb740 legacy=0x7f69ec0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.420+0000 7f69ebfff700 1 -- 172.21.15.92:0/2907085950 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec070e30 con 0x7f69ec15a0f0 2023-06-09T04:45:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.421+0000 7f69ebfff700 1 --2- 172.21.15.92:0/2907085950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec15a0f0 0x7f69ec0c8e00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009840 tx=0x7f69dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6593bbe175da641f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.421+0000 7f69eaffd700 1 -- 172.21.15.92:0/2907085950 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc011830 con 0x7f69ec15a0f0 2023-06-09T04:45:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.421+0000 7f69eaffd700 1 -- 172.21.15.92:0/2907085950 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc011990 con 0x7f69ec15a0f0 2023-06-09T04:45:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.421+0000 7f69eaffd700 1 -- 172.21.15.92:0/2907085950 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc011d20 con 0x7f69ec15a0f0 2023-06-09T04:45:30.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.422+0000 7f69f29cc700 1 -- 172.21.15.92:0/2907085950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec15a0f0 msgr2=0x7f69ec0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.422+0000 7f69f29cc700 1 --2- 172.21.15.92:0/2907085950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec15a0f0 0x7f69ec0c8e00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009840 tx=0x7f69dc009b50 comp rx=0 tx=0).stop 2023-06-09T04:45:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.422+0000 7f69f29cc700 1 -- 172.21.15.92:0/2907085950 shutdown_connections 2023-06-09T04:45:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.422+0000 7f69f29cc700 1 --2- 172.21.15.92:0/2907085950 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec15a0f0 0x7f69ec0c8e00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.422+0000 7f69f29cc700 1 -- 172.21.15.92:0/2907085950 >> 172.21.15.92:0/2907085950 conn(0x7f69ec06aa40 msgr2=0x7f69ec0b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.422+0000 7f69f29cc700 1 -- 172.21.15.92:0/2907085950 shutdown_connections 2023-06-09T04:45:30.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.422+0000 7f69f29cc700 1 -- 172.21.15.92:0/2907085950 wait complete. 2023-06-09T04:45:30.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.424+0000 7f69f29cc700 1 Processor -- start 2023-06-09T04:45:30.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.424+0000 7f69f29cc700 1 -- start start 2023-06-09T04:45:30.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69f29cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec0cb740 0x7f69ec0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:30.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69f29cc700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69ec0ce250 0x7f69ec0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:30.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69f29cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec15a0f0 0x7f69ec0b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:30.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69f29cc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec154fd0 con 0x7f69ec0cb740 2023-06-09T04:45:30.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69f29cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0b3bb0 con 0x7f69ec15a0f0 2023-06-09T04:45:30.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69f29cc700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec15a670 con 0x7f69ec0ce250 2023-06-09T04:45:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec0cb740 0x7f69ec0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec0cb740 0x7f69ec0c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40126/0 (socket says 172.21.15.92:40126) 2023-06-09T04:45:30.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69eb7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69ec0ce250 0x7f69ec0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 -- 172.21.15.92:0/783342609 learned_addr learned my addr 172.21.15.92:0/783342609 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69f0f69700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec15a0f0 0x7f69ec0b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 -- 172.21.15.92:0/783342609 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69ec0ce250 msgr2=0x7f69ec0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69ec0ce250 0x7f69ec0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 -- 172.21.15.92:0/783342609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec15a0f0 msgr2=0x7f69ec0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec15a0f0 0x7f69ec0b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.426+0000 7f69ebfff700 1 -- 172.21.15.92:0/783342609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc0094a0 con 0x7f69ec0cb740 2023-06-09T04:45:30.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.427+0000 7f69ebfff700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec0cb740 0x7f69ec0c4720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f69dc000c00 tx=0x7f69dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.428+0000 7f69e8ff9700 1 -- 172.21.15.92:0/783342609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0045d0 con 0x7f69ec0cb740 2023-06-09T04:45:30.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.429+0000 7f69e8ff9700 1 -- 172.21.15.92:0/783342609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc01e070 con 0x7f69ec0cb740 2023-06-09T04:45:30.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.429+0000 7f69e8ff9700 1 -- 172.21.15.92:0/783342609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0038c0 con 0x7f69ec0cb740 2023-06-09T04:45:30.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.429+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec15a950 con 0x7f69ec0cb740 2023-06-09T04:45:30.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.429+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec15af10 con 0x7f69ec0cb740 2023-06-09T04:45:30.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.430+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec04e8e0 con 0x7f69ec0cb740 2023-06-09T04:45:30.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.430+0000 7f69e8ff9700 1 -- 172.21.15.92:0/783342609 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69dc003a20 con 0x7f69ec0cb740 2023-06-09T04:45:30.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.431+0000 7f69e8ff9700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f69d806ad30 0x7f69d806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:30.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.431+0000 7f69e8ff9700 1 -- 172.21.15.92:0/783342609 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f69dc08a480 con 0x7f69ec0cb740 2023-06-09T04:45:30.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.431+0000 7f69eb7fe700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f69d806ad30 0x7f69d806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:30.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.432+0000 7f69eb7fe700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f69d806ad30 0x7f69d806d1e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0b47d0 tx=0x7f69e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:30.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.436+0000 7f69e8ff9700 1 -- 172.21.15.92:0/783342609 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f69dc055620 con 0x7f69ec0cb740 2023-06-09T04:45:30.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.770+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69ec0b0850 con 0x7f69ec0cb740 2023-06-09T04:45:30.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.771+0000 7f69e8ff9700 1 -- 172.21.15.92:0/783342609 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69dc058c40 con 0x7f69ec0cb740 2023-06-09T04:45:30.781 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-06-09T04:45:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.780+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f69d806ad30 msgr2=0x7f69d806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.780+0000 7f69f29cc700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f69d806ad30 0x7f69d806d1e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0b47d0 tx=0x7f69e0009040 comp rx=0 tx=0).stop 2023-06-09T04:45:30.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.780+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec0cb740 msgr2=0x7f69ec0c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:30.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.780+0000 7f69f29cc700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec0cb740 0x7f69ec0c4720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f69dc000c00 tx=0x7f69dc004750 comp rx=0 tx=0).stop 2023-06-09T04:45:30.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.781+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 shutdown_connections 2023-06-09T04:45:30.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.781+0000 7f69f29cc700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f69d806ad30 0x7f69d806d1e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.781+0000 7f69f29cc700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69ec15a0f0 0x7f69ec0b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.781+0000 7f69f29cc700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69ec0cb740 0x7f69ec0c4720 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.781+0000 7f69f29cc700 1 --2- 172.21.15.92:0/783342609 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f69ec0ce250 0x7f69ec0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.781+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 >> 172.21.15.92:0/783342609 conn(0x7f69ec06aa40 msgr2=0x7f69ec06e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.781+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 shutdown_connections 2023-06-09T04:45:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:30.782+0000 7f69f29cc700 1 -- 172.21.15.92:0/783342609 wait complete. 2023-06-09T04:45:30.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T04:45:31.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.015+0000 7f38d7725700 1 Processor -- start 2023-06-09T04:45:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.015+0000 7f38d7725700 1 -- start start 2023-06-09T04:45:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.016+0000 7f38d7725700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38d00713b0 0x7f38d01596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.016+0000 7f38d7725700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00ce190 con 0x7f38d00aed50 2023-06-09T04:45:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.016+0000 7f38d7725700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00717f0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.016+0000 7f38d7725700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d0072350 con 0x7f38d00713b0 2023-06-09T04:45:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.016+0000 7f38d54c1700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f38d00ce3d0 0x7f38d00ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:60960/0 (socket says 172.21.15.92:60960) 2023-06-09T04:45:31.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38d54c1700 1 -- 172.21.15.92:0/1145479339 learned_addr learned my addr 172.21.15.92:0/1145479339 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:45:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38d5cc2700 1 --2- 172.21.15.92:0/1145479339 >> v2:172.21.15.92:3300/0 conn(0x7f38d00713b0 0x7f38d01596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38d5cc2700 1 -- 172.21.15.92:0/1145479339 >> v1:172.21.15.123:6789/0 conn(0x7f38d00ce3d0 legacy=0x7f38d00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38c7fff700 1 -- 172.21.15.92:0/1145479339 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656948797 0 0) 0x7f38d00717f0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38d5cc2700 1 -- 172.21.15.92:0/1145479339 >> v1:172.21.15.92:6789/0 conn(0x7f38d00aed50 legacy=0x7f38d015d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38d5cc2700 1 -- 172.21.15.92:0/1145479339 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0071ab0 con 0x7f38d00713b0 2023-06-09T04:45:31.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38d5cc2700 1 --2- 172.21.15.92:0/1145479339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00713b0 0x7f38d01596e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009d90 tx=0x7f38c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a383ef65c7a6d8c1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:31.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38c7fff700 1 -- 172.21.15.92:0/1145479339 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800e070 con 0x7f38d00713b0 2023-06-09T04:45:31.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38c7fff700 1 -- 172.21.15.92:0/1145479339 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c800bbd0 con 0x7f38d00713b0 2023-06-09T04:45:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.017+0000 7f38c7fff700 1 -- 172.21.15.92:0/1145479339 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8012430 con 0x7f38d00713b0 2023-06-09T04:45:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.018+0000 7f38d7725700 1 -- 172.21.15.92:0/1145479339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00713b0 msgr2=0x7f38d01596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.018+0000 7f38d7725700 1 --2- 172.21.15.92:0/1145479339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00713b0 0x7f38d01596e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009d90 tx=0x7f38c800b330 comp rx=0 tx=0).stop 2023-06-09T04:45:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.020+0000 7f38d7725700 1 -- 172.21.15.92:0/1145479339 shutdown_connections 2023-06-09T04:45:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.020+0000 7f38d7725700 1 --2- 172.21.15.92:0/1145479339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00713b0 0x7f38d01596e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.020+0000 7f38d7725700 1 -- 172.21.15.92:0/1145479339 >> 172.21.15.92:0/1145479339 conn(0x7f38d00cc380 msgr2=0x7f38d00cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:31.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.029+0000 7f38d7725700 1 -- 172.21.15.92:0/1145479339 shutdown_connections 2023-06-09T04:45:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.033+0000 7f38d7725700 1 -- 172.21.15.92:0/1145479339 wait complete. 2023-06-09T04:45:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.034+0000 7f38d7725700 1 Processor -- start 2023-06-09T04:45:31.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.034+0000 7f38d7725700 1 -- start start 2023-06-09T04:45:31.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d7725700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d00713b0 0x7f38d0151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d7725700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00aed50 0x7f38d00c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d7725700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ce3d0 0x7f38d01e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d7725700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b4380 con 0x7f38d00ce3d0 2023-06-09T04:45:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d7725700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b4660 con 0x7f38d00aed50 2023-06-09T04:45:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d7725700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b3700 con 0x7f38d00713b0 2023-06-09T04:45:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d5cc2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ce3d0 0x7f38d01e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d5cc2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ce3d0 0x7f38d01e7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40142/0 (socket says 172.21.15.92:40142) 2023-06-09T04:45:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d5cc2700 1 -- 172.21.15.92:0/3206206718 learned_addr learned my addr 172.21.15.92:0/3206206718 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d4cc0700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00aed50 0x7f38d00c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d5cc2700 1 -- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d00713b0 msgr2=0x7f38d0151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.035+0000 7f38d5cc2700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d00713b0 0x7f38d0151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38d5cc2700 1 -- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00aed50 msgr2=0x7f38d00c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38d5cc2700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00aed50 0x7f38d00c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38d5cc2700 1 -- 172.21.15.92:0/3206206718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009a70 con 0x7f38d00ce3d0 2023-06-09T04:45:31.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38d4cc0700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00aed50 0x7f38d00c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T04:45:31.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38d5cc2700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ce3d0 0x7f38d01e7d40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f38c8014040 tx=0x7f38c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:31.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38c5ffb700 1 -- 172.21.15.92:0/3206206718 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c801b810 con 0x7f38d00ce3d0 2023-06-09T04:45:31.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38c5ffb700 1 -- 172.21.15.92:0/3206206718 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8020070 con 0x7f38d00ce3d0 2023-06-09T04:45:31.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.036+0000 7f38c5ffb700 1 -- 172.21.15.92:0/3206206718 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c801bbb0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.039+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d00b39e0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.039+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00b3fa0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.040+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d00b44c0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.041+0000 7f38c5ffb700 1 -- 172.21.15.92:0/3206206718 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38c801bd10 con 0x7f38d00ce3d0 2023-06-09T04:45:31.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.041+0000 7f38c5ffb700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f38b406ace0 0x7f38b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.042+0000 7f38c5ffb700 1 -- 172.21.15.92:0/3206206718 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f38c8022070 con 0x7f38d00ce3d0 2023-06-09T04:45:31.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.042+0000 7f38d54c1700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f38b406ace0 0x7f38b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.042+0000 7f38d54c1700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f38b406ace0 0x7f38b406d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f38cc007fd0 tx=0x7f38cc007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:31.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.056+0000 7f38c5ffb700 1 -- 172.21.15.92:0/3206206718 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f38d00b44c0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.365+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38d00b44c0 con 0x7f38d00ce3d0 2023-06-09T04:45:31.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.366+0000 7f38c5ffb700 1 -- 172.21.15.92:0/3206206718 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v41) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38c802c020 con 0x7f38d00ce3d0 2023-06-09T04:45:31.368 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-06-09T04:45:31.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.377+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f38b406ace0 msgr2=0x7f38b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.377+0000 7f38d7725700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f38b406ace0 0x7f38b406d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f38cc007fd0 tx=0x7f38cc007960 comp rx=0 tx=0).stop 2023-06-09T04:45:31.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.377+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ce3d0 msgr2=0x7f38d01e7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.377+0000 7f38d7725700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ce3d0 0x7f38d01e7d40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f38c8014040 tx=0x7f38c8004ad0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.386+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 shutdown_connections 2023-06-09T04:45:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.386+0000 7f38d7725700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f38b406ace0 0x7f38b406d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.386+0000 7f38d7725700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00ce3d0 0x7f38d01e7d40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.386+0000 7f38d7725700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d00713b0 0x7f38d0151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.386+0000 7f38d7725700 1 --2- 172.21.15.92:0/3206206718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d00aed50 0x7f38d00c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.386+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 >> 172.21.15.92:0/3206206718 conn(0x7f38d00cc380 msgr2=0x7f38d00af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:31.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.390+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 shutdown_connections 2023-06-09T04:45:31.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.390+0000 7f38d7725700 1 -- 172.21.15.92:0/3206206718 wait complete. 2023-06-09T04:45:31.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T04:45:31.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.617+0000 7fbbdb5be700 1 Processor -- start 2023-06-09T04:45:31.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.618+0000 7fbbdb5be700 1 -- start start 2023-06-09T04:45:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbdb5be700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbd406d2c0 0x7fbbd40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbdb5be700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd406adc0 con 0x7fbbd415b180 2023-06-09T04:45:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbdb5be700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd4071ab0 con 0x7fbbd40716e0 2023-06-09T04:45:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbdb5be700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd40cee80 con 0x7fbbd406d2c0 2023-06-09T04:45:31.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbd935a700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbbd40716e0 0x7fbbd406cbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:60974/0 (socket says 172.21.15.92:60974) 2023-06-09T04:45:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbd935a700 1 -- 172.21.15.92:0/1619290239 learned_addr learned my addr 172.21.15.92:0/1619290239 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-06-09T04:45:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118709772 0 0) 0x7fbbd4071ab0 con 0x7fbbd40716e0 2023-06-09T04:45:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.619+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc00035b0 con 0x7fbbd40716e0 2023-06-09T04:45:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536241304 0 0) 0x7fbbc00035b0 con 0x7fbbd40716e0 2023-06-09T04:45:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 >> v2:172.21.15.123:3300/0 conn(0x7fbbd406d2c0 msgr2=0x7fbbd40ce7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:45:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 --2- 172.21.15.92:0/1619290239 >> v2:172.21.15.123:3300/0 conn(0x7fbbd406d2c0 0x7fbbd40ce7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 >> v1:172.21.15.92:6789/0 conn(0x7fbbd415b180 legacy=0x7fbbd4163d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd406b300 con 0x7fbbd40716e0 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3756822464 0 0) 0x7fbbd0001670 con 0x7fbbd40716e0 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 >> v1:172.21.15.123:6789/0 conn(0x7fbbd40716e0 legacy=0x7fbbd406cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc0004850 0x7fbbc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbc0005320 0x7fbbc000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40cee80 con 0x7fbbc0003a00 2023-06-09T04:45:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc000c1e0 con 0x7fbbc0004850 2023-06-09T04:45:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc000c3a0 con 0x7fbbc0005320 2023-06-09T04:45:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbd935a700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbd935a700 1 -- 172.21.15.92:0/1619290239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbc0005320 msgr2=0x7fbbc000ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:45:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbd935a700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbc0005320 0x7fbbc000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbd935a700 1 -- 172.21.15.92:0/1619290239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc0004850 msgr2=0x7fbbc0004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T04:45:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbd935a700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc0004850 0x7fbbc0004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbd935a700 1 -- 172.21.15.92:0/1619290239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd406b300 con 0x7fbbc0003a00 2023-06-09T04:45:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.620+0000 7fbbd935a700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003df0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c370 tx=0x7fbbc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ddf317cd915f60 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.621+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0005960 con 0x7fbbc0003a00 2023-06-09T04:45:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.621+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0005ac0 con 0x7fbbc0003a00 2023-06-09T04:45:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.621+0000 7fbbcbfff700 1 -- 172.21.15.92:0/1619290239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0005e70 con 0x7fbbc0003a00 2023-06-09T04:45:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.622+0000 7fbbd9b5b700 1 -- 172.21.15.92:0/1619290239 reap_dead start 2023-06-09T04:45:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.623+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1619290239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0003a00 msgr2=0x7fbbc0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.623+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003df0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c370 tx=0x7fbbc0005880 comp rx=0 tx=0).stop 2023-06-09T04:45:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.623+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1619290239 shutdown_connections 2023-06-09T04:45:31.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.623+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1619290239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0003a00 0x7fbbc0003df0 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000c370 tx=0x7fbbc0005880 comp rx=0 tx=0).stop 2023-06-09T04:45:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.623+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1619290239 >> 172.21.15.92:0/1619290239 conn(0x7fbbd40cfc40 msgr2=0x7fbbd40b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.623+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1619290239 shutdown_connections 2023-06-09T04:45:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.623+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1619290239 wait complete. 2023-06-09T04:45:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.625+0000 7fbbdb5be700 1 Processor -- start 2023-06-09T04:45:31.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.625+0000 7fbbdb5be700 1 -- start start 2023-06-09T04:45:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbdb5be700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd4153660 0x7fbbd40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbdb5be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd4153a30 0x7fbbd41e3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbdb5be700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbd40c3e40 0x7fbbd40c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbdb5be700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd41546f0 con 0x7fbbd4153660 2023-06-09T04:45:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbdb5be700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40c5920 con 0x7fbbd4153a30 2023-06-09T04:45:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbdb5be700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40c4340 con 0x7fbbd40c3e40 2023-06-09T04:45:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd4153660 0x7fbbd40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd4153660 0x7fbbd40c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40150/0 (socket says 172.21.15.92:40150) 2023-06-09T04:45:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 -- 172.21.15.92:0/1025702850 learned_addr learned my addr 172.21.15.92:0/1025702850 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd9b5b700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbd40c3e40 0x7fbbd40c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 -- 172.21.15.92:0/1025702850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbd40c3e40 msgr2=0x7fbbd40c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbd40c3e40 0x7fbbd40c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 -- 172.21.15.92:0/1025702850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd4153a30 msgr2=0x7fbbd41e3980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd4153a30 0x7fbbd41e3980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 -- 172.21.15.92:0/1025702850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd40c4620 con 0x7fbbd4153660 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.626+0000 7fbbd935a700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd4153660 0x7fbbd40c53e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c1b0 tx=0x7fbbd0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.627+0000 7fbbc9ffb700 1 -- 172.21.15.92:0/1025702850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000cee0 con 0x7fbbd4153660 2023-06-09T04:45:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.627+0000 7fbbc9ffb700 1 -- 172.21.15.92:0/1025702850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0004a80 con 0x7fbbd4153660 2023-06-09T04:45:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.627+0000 7fbbc9ffb700 1 -- 172.21.15.92:0/1025702850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0004df0 con 0x7fbbd4153660 2023-06-09T04:45:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.627+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd40c4900 con 0x7fbbd4153660 2023-06-09T04:45:31.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.627+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd40c4ec0 con 0x7fbbd4153660 2023-06-09T04:45:31.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.633+0000 7fbbc9ffb700 1 -- 172.21.15.92:0/1025702850 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbbd00070d0 con 0x7fbbd4153660 2023-06-09T04:45:31.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.633+0000 7fbbc9ffb700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbb806f170 0x7fbbb8071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:31.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.633+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb40052f0 con 0x7fbbd4153660 2023-06-09T04:45:31.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.633+0000 7fbbd8b59700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbb806f170 0x7fbbb8071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:31.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.634+0000 7fbbc9ffb700 1 -- 172.21.15.92:0/1025702850 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fbbd005c920 con 0x7fbbd4153660 2023-06-09T04:45:31.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.634+0000 7fbbd8b59700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbb806f170 0x7fbbb8071620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbd41f3290 tx=0x7fbbc4006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.640+0000 7fbbc9ffb700 1 -- 172.21.15.92:0/1025702850 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbbd0054ca0 con 0x7fbbd4153660 2023-06-09T04:45:31.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.962+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbb4005b20 con 0x7fbbd4153660 2023-06-09T04:45:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.963+0000 7fbbc9ffb700 1 -- 172.21.15.92:0/1025702850 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbd00582c0 con 0x7fbbd4153660 2023-06-09T04:45:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-06-09T04:45:31.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.967+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbb806f170 msgr2=0x7fbbb8071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.967+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbb806f170 0x7fbbb8071620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbd41f3290 tx=0x7fbbc4006bf0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.967+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd4153660 msgr2=0x7fbbd40c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.967+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd4153660 0x7fbbd40c53e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000c1b0 tx=0x7fbbd0001930 comp rx=0 tx=0).stop 2023-06-09T04:45:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.970+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 shutdown_connections 2023-06-09T04:45:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.970+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7fbbb806f170 0x7fbbb8071620 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.970+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbd40c3e40 0x7fbbd40c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.970+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbd4153660 0x7fbbd40c53e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.970+0000 7fbbdb5be700 1 --2- 172.21.15.92:0/1025702850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd4153a30 0x7fbbd41e3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.971+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 >> 172.21.15.92:0/1025702850 conn(0x7fbbd40cfc40 msgr2=0x7fbbd40b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:31.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.971+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 shutdown_connections 2023-06-09T04:45:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:31.971+0000 7fbbdb5be700 1 -- 172.21.15.92:0/1025702850 wait complete. 2023-06-09T04:45:31.987 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T04:45:32.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.189+0000 7f3216c99700 1 Processor -- start 2023-06-09T04:45:32.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.189+0000 7f3216c99700 1 -- start start 2023-06-09T04:45:32.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f3216c99700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f32100ce3d0 0x7f32100713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f3216c99700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32100aed50 0x7f32100ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f3216c99700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100ce190 con 0x7f321015d470 2023-06-09T04:45:32.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f3216c99700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3210072350 con 0x7f32100aed50 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f3216c99700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100cece0 con 0x7f32100ce3d0 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32100aed50 0x7f32100ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f32100aed50 0x7f32100ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40158/0 (socket says 172.21.15.92:40158) 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 -- 172.21.15.92:0/3262344927 learned_addr learned my addr 172.21.15.92:0/3262344927 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f3214a35700 1 --2- 172.21.15.92:0/3262344927 >> v2:172.21.15.123:3300/0 conn(0x7f32100ce3d0 0x7f32100713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 -- 172.21.15.92:0/3262344927 >> v2:172.21.15.123:3300/0 conn(0x7f32100ce3d0 msgr2=0x7f32100713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 --2- 172.21.15.92:0/3262344927 >> v2:172.21.15.123:3300/0 conn(0x7f32100ce3d0 0x7f32100713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 -- 172.21.15.92:0/3262344927 >> v1:172.21.15.123:6789/0 conn(0x7f321015d470 legacy=0x7f32101596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 -- 172.21.15.92:0/3262344927 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210071ab0 con 0x7f32100aed50 2023-06-09T04:45:32.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320ffff700 1 --2- 172.21.15.92:0/3262344927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32100ce7a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f321006fee0 tx=0x7f31fc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6a23a590ed0248 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:32.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.190+0000 7f320f7fe700 1 -- 172.21.15.92:0/3262344927 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00dc70 con 0x7f32100aed50 2023-06-09T04:45:32.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f320f7fe700 1 -- 172.21.15.92:0/3262344927 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc00ddd0 con 0x7f32100aed50 2023-06-09T04:45:32.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f320f7fe700 1 -- 172.21.15.92:0/3262344927 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc010690 con 0x7f32100aed50 2023-06-09T04:45:32.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 -- 172.21.15.92:0/3262344927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 msgr2=0x7f32100ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 --2- 172.21.15.92:0/3262344927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32100ce7a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f321006fee0 tx=0x7f31fc00be70 comp rx=0 tx=0).stop 2023-06-09T04:45:32.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 -- 172.21.15.92:0/3262344927 shutdown_connections 2023-06-09T04:45:32.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 --2- 172.21.15.92:0/3262344927 >> v2:172.21.15.123:3300/0 conn(0x7f32100ce3d0 0x7f32100713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 --2- 172.21.15.92:0/3262344927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32100ce7a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 -- 172.21.15.92:0/3262344927 >> 172.21.15.92:0/3262344927 conn(0x7f32100cc380 msgr2=0x7f32100cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:32.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 -- 172.21.15.92:0/3262344927 shutdown_connections 2023-06-09T04:45:32.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.191+0000 7f3216c99700 1 -- 172.21.15.92:0/3262344927 wait complete. 2023-06-09T04:45:32.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.193+0000 7f3216c99700 1 Processor -- start 2023-06-09T04:45:32.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.193+0000 7f3216c99700 1 -- start start 2023-06-09T04:45:32.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3216c99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32101f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3216c99700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32100ce3d0 0x7f321014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3216c99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f321015d470 0x7f32100b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3216c99700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32101f2600 con 0x7f32100aed50 2023-06-09T04:45:32.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3216c99700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321014d5b0 con 0x7f321015d470 2023-06-09T04:45:32.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3216c99700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100c6070 con 0x7f32100ce3d0 2023-06-09T04:45:32.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3214a35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32101f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3214a35700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32101f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40162/0 (socket says 172.21.15.92:40162) 2023-06-09T04:45:32.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.194+0000 7f3214a35700 1 -- 172.21.15.92:0/1244562545 learned_addr learned my addr 172.21.15.92:0/1244562545 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:32.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3214a35700 1 -- 172.21.15.92:0/1244562545 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32100ce3d0 msgr2=0x7f321014ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3214a35700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32100ce3d0 0x7f321014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3214a35700 1 -- 172.21.15.92:0/1244562545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f321015d470 msgr2=0x7f32100b4f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:45:32.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3214a35700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f321015d470 0x7f32100b4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3214a35700 1 -- 172.21.15.92:0/1244562545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc00b7f0 con 0x7f32100aed50 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3214a35700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32101f20c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f320400e9b0 tx=0x7f320400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f320d7fa700 1 -- 172.21.15.92:0/1244562545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320400c850 con 0x7f32100aed50 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f320d7fa700 1 -- 172.21.15.92:0/1244562545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320400c9b0 con 0x7f32100aed50 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32100b4430 con 0x7f32100aed50 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f320d7fa700 1 -- 172.21.15.92:0/1244562545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320400cd20 con 0x7f32100aed50 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.195+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100b49f0 con 0x7f32100aed50 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.197+0000 7f320d7fa700 1 -- 172.21.15.92:0/1244562545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3204013470 con 0x7f32100aed50 2023-06-09T04:45:32.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.198+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32100b4c10 con 0x7f32100aed50 2023-06-09T04:45:32.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.198+0000 7f320d7fa700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.198+0000 7f320d7fa700 1 -- 172.21.15.92:0/1244562545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f32040896a0 con 0x7f32100aed50 2023-06-09T04:45:32.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.205+0000 7f320ffff700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.206+0000 7f320d7fa700 1 -- 172.21.15.92:0/1244562545 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f32040546e0 con 0x7f32100aed50 2023-06-09T04:45:32.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.206+0000 7f320ffff700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31f406ace0 0x7f31f406d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32100c5a80 tx=0x7f31fc012000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.524+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f321006be50 con 0x7f32100aed50 2023-06-09T04:45:32.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.524+0000 7f320d7fa700 1 -- 172.21.15.92:0/1244562545 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3204057d00 con 0x7f32100aed50 2023-06-09T04:45:32.525 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-06-09T04:45:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31f406ace0 msgr2=0x7f31f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31f406ace0 0x7f31f406d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32100c5a80 tx=0x7f31fc012000 comp rx=0 tx=0).stop 2023-06-09T04:45:32.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 msgr2=0x7f32101f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32101f20c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f320400e9b0 tx=0x7f320400ed70 comp rx=0 tx=0).stop 2023-06-09T04:45:32.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 shutdown_connections 2023-06-09T04:45:32.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f31f406ace0 0x7f31f406d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f321015d470 0x7f32100b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32100aed50 0x7f32101f20c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 --2- 172.21.15.92:0/1244562545 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f32100ce3d0 0x7f321014ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 >> 172.21.15.92:0/1244562545 conn(0x7f32100cc380 msgr2=0x7f32100cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:32.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.527+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 shutdown_connections 2023-06-09T04:45:32.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.528+0000 7f3216c99700 1 -- 172.21.15.92:0/1244562545 wait complete. 2023-06-09T04:45:32.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-09T04:45:32.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.745+0000 7f8f41dee700 1 Processor -- start 2023-06-09T04:45:32.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.745+0000 7f8f41dee700 1 -- start start 2023-06-09T04:45:32.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.746+0000 7f8f41dee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c15a380 0x7f8f3c0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.746+0000 7f8f41dee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f3c0afef0 0x7f8f3c0cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.746+0000 7f8f41dee700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0cdd10 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.746+0000 7f8f41dee700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c06eca0 con 0x7f8f3c0afef0 2023-06-09T04:45:32.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.746+0000 7f8f41dee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0d07f0 con 0x7f8f3c15a380 2023-06-09T04:45:32.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.746+0000 7f8f3b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c15a380 0x7f8f3c0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.746+0000 7f8f3affd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f3c0afef0 0x7f8f3c0cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c15a380 0x7f8f3c0ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:49078/0 (socket says 172.21.15.92:49078) 2023-06-09T04:45:32.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/521920160 learned_addr learned my addr 172.21.15.92:0/521920160 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:32.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3bfff700 1 --1- 172.21.15.92:0/521920160 >> v1:172.21.15.123:6789/0 conn(0x7f8f3c0af9a0 0x7f8f3c06f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.92:32768/0 (socket says 172.21.15.92:32768) 2023-06-09T04:45:32.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/521920160 >> v2:172.21.15.123:3300/0 conn(0x7f8f3c0afef0 msgr2=0x7f8f3c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T04:45:32.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3b7fe700 1 --2- 172.21.15.92:0/521920160 >> v2:172.21.15.123:3300/0 conn(0x7f8f3c0afef0 0x7f8f3c0cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/521920160 >> v1:172.21.15.123:6789/0 conn(0x7f8f3c0af9a0 legacy=0x7f8f3c06f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/521920160 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c15a750 con 0x7f8f3c15a380 2023-06-09T04:45:32.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3affd700 1 --2- 172.21.15.92:0/521920160 >> v2:172.21.15.123:3300/0 conn(0x7f8f3c0afef0 0x7f8f3c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T04:45:32.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3b7fe700 1 --2- 172.21.15.92:0/521920160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c15a380 0x7f8f3c0ce540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f300098f0 tx=0x7f8f30009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b2b11234461f6f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:32.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3a7fc700 1 -- 172.21.15.92:0/521920160 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30011830 con 0x7f8f3c15a380 2023-06-09T04:45:32.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3a7fc700 1 -- 172.21.15.92:0/521920160 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30011990 con 0x7f8f3c15a380 2023-06-09T04:45:32.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.747+0000 7f8f3a7fc700 1 -- 172.21.15.92:0/521920160 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30011d20 con 0x7f8f3c15a380 2023-06-09T04:45:32.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.748+0000 7f8f41dee700 1 -- 172.21.15.92:0/521920160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c15a380 msgr2=0x7f8f3c0ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.748+0000 7f8f41dee700 1 --2- 172.21.15.92:0/521920160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c15a380 0x7f8f3c0ce540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f300098f0 tx=0x7f8f30009c00 comp rx=0 tx=0).stop 2023-06-09T04:45:32.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.748+0000 7f8f41dee700 1 -- 172.21.15.92:0/521920160 shutdown_connections 2023-06-09T04:45:32.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.748+0000 7f8f41dee700 1 --2- 172.21.15.92:0/521920160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c15a380 0x7f8f3c0ce540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.748+0000 7f8f41dee700 1 --2- 172.21.15.92:0/521920160 >> v2:172.21.15.123:3300/0 conn(0x7f8f3c0afef0 0x7f8f3c0cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.748+0000 7f8f41dee700 1 -- 172.21.15.92:0/521920160 >> 172.21.15.92:0/521920160 conn(0x7f8f3c089fe0 msgr2=0x7f8f3c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:32.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.756+0000 7f8f41dee700 1 -- 172.21.15.92:0/521920160 shutdown_connections 2023-06-09T04:45:32.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.758+0000 7f8f41dee700 1 -- 172.21.15.92:0/521920160 wait complete. 2023-06-09T04:45:32.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.759+0000 7f8f41dee700 1 Processor -- start 2023-06-09T04:45:32.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.760+0000 7f8f41dee700 1 -- start start 2023-06-09T04:45:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f41dee700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c0af9a0 0x7f8f3c2023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f41dee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c0afef0 0x7f8f3c2028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f41dee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f3c15a380 0x7f8f3c0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f41dee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b3390 con 0x7f8f3c15a380 2023-06-09T04:45:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f41dee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b1470 con 0x7f8f3c0afef0 2023-06-09T04:45:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f41dee700 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b1750 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c0af9a0 0x7f8f3c2023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c0af9a0 0x7f8f3c2023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.92:49084/0 (socket says 172.21.15.92:49084) 2023-06-09T04:45:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/999105981 learned_addr learned my addr 172.21.15.92:0/999105981 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-06-09T04:45:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3affd700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c0afef0 0x7f8f3c2028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c0afef0 msgr2=0x7f8f3c2028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c0afef0 0x7f8f3c2028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f3c15a380 msgr2=0x7f8f3c0b0870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T04:45:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f3c15a380 0x7f8f3c0b0870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.761+0000 7f8f3b7fe700 1 -- 172.21.15.92:0/999105981 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c0094a0 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.762+0000 7f8f3b7fe700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c0af9a0 0x7f8f3c2023b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f30014040 tx=0x7f8f30004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.762+0000 7f8f23fff700 1 -- 172.21.15.92:0/999105981 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30004590 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.762+0000 7f8f23fff700 1 -- 172.21.15.92:0/999105981 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3001e070 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.762+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f30009550 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.762+0000 7f8f23fff700 1 -- 172.21.15.92:0/999105981 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f300038a0 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.762+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c0b1d10 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.763+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c0740e0 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.764+0000 7f8f23fff700 1 -- 172.21.15.92:0/999105981 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f30003a00 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.764+0000 7f8f23fff700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8f2806ace0 0x7f8f2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T04:45:32.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.764+0000 7f8f23fff700 1 -- 172.21.15.92:0/999105981 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f8f3008a430 con 0x7f8f3c0af9a0 2023-06-09T04:45:32.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.775+0000 7f8f3affd700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8f2806ace0 0x7f8f2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T04:45:32.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.776+0000 7f8f3affd700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8f2806ace0 0x7f8f2806d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0caa80 tx=0x7f8f2c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T04:45:32.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:32.779+0000 7f8f23fff700 1 -- 172.21.15.92:0/999105981 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f300554f0 con 0x7f8f3c0af9a0 2023-06-09T04:45:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.147+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8f3c0c33c0 con 0x7f8f3c0af9a0 2023-06-09T04:45:33.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.155+0000 7f8f23fff700 1 -- 172.21.15.92:0/999105981 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v41) v1 ==== 80+0+120 (secure 0 0 0) 0x7f8f30058b10 con 0x7f8f3c0af9a0 2023-06-09T04:45:33.156 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6837/637518437 2023-06-10T04:40:32.122135+0000 2023-06-09T04:45:33.157 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92:6835/637518437 2023-06-10T04:40:32.122135+0000 2023-06-09T04:45:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8f2806ace0 msgr2=0x7f8f2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8f2806ace0 0x7f8f2806d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0caa80 tx=0x7f8f2c009210 comp rx=0 tx=0).stop 2023-06-09T04:45:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c0af9a0 msgr2=0x7f8f3c2023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T04:45:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c0af9a0 0x7f8f3c2023b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f30014040 tx=0x7f8f30004750 comp rx=0 tx=0).stop 2023-06-09T04:45:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 shutdown_connections 2023-06-09T04:45:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:6832/3112104013,v1:172.21.15.92:6833/3112104013] conn(0x7f8f2806ace0 0x7f8f2806d190 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0caa80 tx=0x7f8f2c009210 comp rx=0 tx=0).stop 2023-06-09T04:45:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f3c15a380 0x7f8f3c0b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8f3c0af9a0 0x7f8f3c2023b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 --2- 172.21.15.92:0/999105981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c0afef0 0x7f8f3c2028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T04:45:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 >> 172.21.15.92:0/999105981 conn(0x7f8f3c089fe0 msgr2=0x7f8f3c06f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T04:45:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 shutdown_connections 2023-06-09T04:45:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-06-09T04:45:33.158+0000 7f8f41dee700 1 -- 172.21.15.92:0/999105981 wait complete. 2023-06-09T04:45:33.161 INFO:teuthology.orchestra.run.smithi092.stderr:listed 2 entries 2023-06-09T04:45:33.173 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-06-09T04:45:33.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:45:33.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-06-09T04:45:33.243 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:45:33.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:45:33.244 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-09T04:45:33.263 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-06-09T04:45:33.263 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:45:33.337 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3710 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 172.21.15.92:6789:/test /tmp/tmp.dvoqC6rw3jtest ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=064bde79-2943-47e5-b941-05f234bd883f,acl 0 0 2023-06-09T04:45:33.337 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-06-09T04:45:33.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:45:33.338 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-09T04:45:33.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:45:33.426 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-06-09T04:45:33.427 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:45:33.427 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:45:33.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4043 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-09T04:45:33.460 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:45:33.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:45:33.460 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-09T04:45:33.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:45:33.548 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-06-09T04:45:33.549 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-06-09T04:45:33.549 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T04:45:33.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4043 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-09T04:45:33.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-06-09T04:45:33.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T04:45:33.636 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-09T04:45:33.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:45:33.684 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-06-09T04:45:35.271 INFO:teuthology.orchestra.run.smithi092.stdout:generic/014 261s 2023-06-09T04:45:35.277 INFO:teuthology.orchestra.run.smithi092.stdout:Ran: generic/014 2023-06-09T04:45:35.278 INFO:teuthology.orchestra.run.smithi092.stdout:Passed all 1 tests 2023-06-09T04:45:35.280 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-06-09T04:45:35.294 INFO:tasks.ceph.osd.0:Sent signal 15 2023-06-09T04:45:35.295 INFO:tasks.ceph.osd.1:Sent signal 15 2023-06-09T04:45:35.295 INFO:tasks.ceph.osd.2:Sent signal 15 2023-06-09T04:45:35.296 INFO:tasks.ceph.osd.3:Sent signal 15 2023-06-09T04:45:35.296 INFO:tasks.ceph.osd.4:Sent signal 15 2023-06-09T04:45:35.296 INFO:tasks.ceph.osd.5:Sent signal 15 2023-06-09T04:45:35.296 INFO:tasks.ceph.osd.6:Sent signal 15 2023-06-09T04:45:35.297 INFO:tasks.ceph.osd.7:Sent signal 15 2023-06-09T04:45:35.297 INFO:tasks.ceph.mds.c:Sent signal 15 2023-06-09T04:45:35.297 INFO:tasks.ceph.mds.b:Sent signal 15 2023-06-09T04:45:35.298 INFO:tasks.ceph.mds.d:Sent signal 15 2023-06-09T04:45:35.298 INFO:tasks.ceph.mon.a:Sent signal 15 2023-06-09T04:45:35.298 INFO:tasks.ceph.mon.b:Sent signal 15 2023-06-09T04:45:35.298 INFO:tasks.ceph.mon.c:Sent signal 15 2023-06-09T04:45:35.299 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-06-09T04:45:35.299 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-06-09T04:45:35.299 INFO:tasks.ceph.mds.b.smithi123.stderr:2023-06-09T04:45:35.297+0000 7f5c00f7e700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 108047) UID: 0 2023-06-09T04:45:35.300 INFO:tasks.ceph.mds.b.smithi123.stderr:2023-06-09T04:45:35.297+0000 7f5c00f7e700 -1 mds.b *** got signal Terminated *** 2023-06-09T04:45:35.300 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-06-09T04:45:35.298+0000 7f02db698700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 100571) UID: 0 2023-06-09T04:45:35.300 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-06-09T04:45:35.298+0000 7f02db698700 -1 osd.4 41 *** Got signal Terminated *** 2023-06-09T04:45:35.300 INFO:tasks.ceph.osd.4.smithi123.stderr:2023-06-09T04:45:35.298+0000 7f02db698700 -1 osd.4 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.301 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-06-09T04:45:35.297+0000 7fef64a33700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 100579) UID: 0 2023-06-09T04:45:35.301 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-06-09T04:45:35.297+0000 7fef64a33700 -1 osd.7 41 *** Got signal Terminated *** 2023-06-09T04:45:35.301 INFO:tasks.ceph.osd.7.smithi123.stderr:2023-06-09T04:45:35.297+0000 7fef64a33700 -1 osd.7 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.301 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-06-09T04:45:35.299+0000 7fa61d33c700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 100577) UID: 0 2023-06-09T04:45:35.302 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-06-09T04:45:35.299+0000 7fa61d33c700 -1 osd.6 41 *** Got signal Terminated *** 2023-06-09T04:45:35.302 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-06-09T04:45:35.297+0000 7f6b0af89700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 100572) UID: 0 2023-06-09T04:45:35.302 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-06-09T04:45:35.297+0000 7f6b0af89700 -1 osd.5 41 *** Got signal Terminated *** 2023-06-09T04:45:35.302 INFO:tasks.ceph.osd.5.smithi123.stderr:2023-06-09T04:45:35.297+0000 7f6b0af89700 -1 osd.5 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.303 INFO:tasks.ceph.osd.6.smithi123.stderr:2023-06-09T04:45:35.299+0000 7fa61d33c700 -1 osd.6 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.303 INFO:tasks.ceph.mon.b.smithi123.stderr:2023-06-09T04:45:35.299+0000 7fc8f65ae700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 99920) UID: 0 2023-06-09T04:45:35.303 INFO:tasks.ceph.mon.b.smithi123.stderr:2023-06-09T04:45:35.299+0000 7fc8f65ae700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-06-09T04:45:35.303 INFO:tasks.ceph.mon.c.smithi123.stderr:2023-06-09T04:45:35.299+0000 7fc8bb77d700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 99922) UID: 0 2023-06-09T04:45:35.304 INFO:tasks.ceph.mon.c.smithi123.stderr:2023-06-09T04:45:35.299+0000 7fc8bb77d700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-06-09T04:45:35.304 INFO:tasks.ceph.mds.d.smithi123.stderr:2023-06-09T04:45:35.299+0000 7f10bce6d700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 108048) UID: 0 2023-06-09T04:45:35.304 INFO:tasks.ceph.mds.d.smithi123.stderr:2023-06-09T04:45:35.299+0000 7f10bce6d700 -1 mds.d *** got signal Terminated *** 2023-06-09T04:45:35.305 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-06-09T04:45:35.294+0000 7f4819180700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 100347) UID: 0 2023-06-09T04:45:35.305 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-06-09T04:45:35.294+0000 7f4819180700 -1 osd.0 41 *** Got signal Terminated *** 2023-06-09T04:45:35.305 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-06-09T04:45:35.294+0000 7f4819180700 -1 osd.0 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.305 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-06-09T04:45:35.295+0000 7f988c780700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 100350) UID: 0 2023-06-09T04:45:35.305 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-06-09T04:45:35.295+0000 7f988c780700 -1 osd.1 41 *** Got signal Terminated *** 2023-06-09T04:45:35.306 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-06-09T04:45:35.295+0000 7f988c780700 -1 osd.1 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.306 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-06-09T04:45:35.296+0000 7f1e40279700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 100353) UID: 0 2023-06-09T04:45:35.306 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-06-09T04:45:35.296+0000 7f1e40279700 -1 osd.2 41 *** Got signal Terminated *** 2023-06-09T04:45:35.306 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-06-09T04:45:35.296+0000 7f1e40279700 -1 osd.2 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.307 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-06-09T04:45:35.301+0000 7f9556aeb700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 100355) UID: 0 2023-06-09T04:45:35.307 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-06-09T04:45:35.301+0000 7f9556aeb700 -1 osd.3 41 *** Got signal Terminated *** 2023-06-09T04:45:35.307 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-06-09T04:45:35.301+0000 7f9556aeb700 -1 osd.3 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-09T04:45:35.307 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-06-09T04:45:35.301+0000 7f45dc145700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 100026) UID: 0 2023-06-09T04:45:35.308 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-06-09T04:45:35.301+0000 7f45dc145700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-06-09T04:45:35.308 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-06-09T04:45:35.302+0000 7f1873dbf700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 113922) UID: 0 2023-06-09T04:45:35.308 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-06-09T04:45:35.302+0000 7f1873dbf700 -1 mds.c *** got signal Terminated *** 2023-06-09T04:45:35.500 INFO:tasks.ceph.mgr.x.smithi123.stderr:daemon-helper: command crashed with signal 15 2023-06-09T04:45:50.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:45:50.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:46:20.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:46:20.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:46:37.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-09T04:46:37.433 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2023-06-09T04:46:50.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:46:50.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:47:20.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:47:20.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:47:38.893 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:47:38.909 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T04:47:50.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:47:50.426 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:48:20.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:48:20.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:48:40.326 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:48:40.328 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T04:48:40.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:48:40.330 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T04:48:40.331 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T04:48:40.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2023-06-09T04:48:50.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:48:50.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:49:20.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:49:20.931 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:49:41.766 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:49:41.768 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T04:49:51.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:49:51.006 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:50:21.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:50:21.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:50:43.207 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:50:43.208 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T04:50:43.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:50:43.212 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T04:50:43.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T04:50:43.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2023-06-09T04:50:51.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:50:51.173 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:51:21.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:51:21.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:51:44.646 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:51:44.648 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T04:51:51.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:51:51.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:52:21.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:52:21.394 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:52:46.087 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:52:46.088 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T04:52:46.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:52:46.091 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T04:52:46.091 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T04:52:46.092 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2023-06-09T04:52:51.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:52:51.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:53:21.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:53:21.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:53:47.527 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:53:47.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T04:53:51.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:53:51.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:54:21.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:54:21.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:54:48.967 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:54:48.969 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T04:54:48.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:54:48.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T04:54:48.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T04:54:48.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2023-06-09T04:54:52.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:54:52.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:55:22.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:55:22.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:55:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:55:50.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T04:55:52.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:55:52.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:56:22.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:56:22.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:56:51.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:56:51.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:56:51.861 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T04:56:51.862 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T04:56:51.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T04:56:51.862 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2023-06-09T04:56:52.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:56:52.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:57:22.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:57:22.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:57:52.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:57:52.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:57:53.287 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:57:53.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T04:58:22.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:58:22.582 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:58:52.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:58:52.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:58:54.727 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:58:54.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T04:58:54.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T04:58:54.731 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T04:58:54.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T04:58:54.732 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2023-06-09T04:59:22.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:59:22.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:59:52.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:59:52.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T04:59:56.167 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T04:59:56.169 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:00:22.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:00:22.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:00:53.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:00:53.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:00:57.607 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:00:57.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:00:57.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:00:57.611 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:00:57.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:00:57.612 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2023-06-09T05:01:23.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:01:23.252 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:01:53.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:01:53.321 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:01:59.047 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:01:59.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:02:23.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:02:23.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:02:53.485 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:02:53.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:03:00.471 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:03:00.473 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:03:00.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:03:00.477 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:03:00.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:03:00.477 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2023-06-09T05:03:23.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:03:23.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:03:53.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:03:53.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:04:01.927 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:04:01.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:04:23.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:04:23.867 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:04:53.944 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:04:53.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:05:03.367 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:05:03.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:05:03.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:05:03.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:05:03.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:05:03.372 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2023-06-09T05:05:24.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:05:24.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:05:54.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:05:54.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:06:04.807 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:06:04.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:06:24.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:06:24.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:06:54.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:06:54.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:07:06.247 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:07:06.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:07:06.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:07:06.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:07:06.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:07:06.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2023-06-09T05:07:24.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:07:24.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:07:54.571 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:07:54.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:08:07.687 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:08:07.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:08:24.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:08:24.770 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:08:54.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:08:54.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:09:09.127 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:09:09.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:09:09.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:09:09.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:09:09.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:09:09.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2023-06-09T05:09:24.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:09:24.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:09:55.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:09:55.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:10:10.567 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:10:10.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:10:25.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:10:25.103 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:10:55.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:10:55.166 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:11:12.007 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:11:12.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:11:12.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:11:12.012 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:11:12.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:11:12.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2023-06-09T05:11:25.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:11:25.237 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:11:55.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:11:55.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:12:13.447 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:12:13.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:12:25.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:12:25.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:12:55.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:12:55.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:13:14.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:13:14.911 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:13:14.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:13:14.914 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:13:14.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:13:14.915 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2023-06-09T05:13:25.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:13:25.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:13:55.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:13:55.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:14:16.327 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:14:16.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:14:25.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:14:25.988 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:14:56.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:14:56.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:15:17.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:15:17.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:15:17.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:15:17.772 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:15:17.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:15:17.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2023-06-09T05:15:26.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:15:26.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:15:56.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:15:56.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:16:19.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:16:19.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:16:26.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:16:26.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:16:56.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:16:56.732 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:17:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:17:20.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:17:20.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:17:20.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:17:20.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:17:20.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2023-06-09T05:17:26.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:17:26.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:17:56.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:17:56.873 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:18:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:18:22.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:18:27.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:18:27.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:18:57.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:18:57.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:19:23.527 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:19:23.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:19:23.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:19:23.532 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:19:23.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:19:23.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2023-06-09T05:19:27.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:19:27.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:19:57.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:19:57.326 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:20:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:20:24.969 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:20:27.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:20:27.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:20:57.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:20:57.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:21:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:21:26.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:21:26.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:21:26.412 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:21:26.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:21:26.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2023-06-09T05:21:27.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:21:27.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:21:57.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:21:57.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:22:27.848 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:22:27.849 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:22:28.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:22:28.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:22:58.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:22:58.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:23:28.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:23:28.296 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:23:29.287 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:23:29.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:23:29.293 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:23:29.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:23:29.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2023-06-09T05:23:58.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:23:58.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:24:28.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:24:28.453 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:24:30.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:24:30.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:24:58.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:24:58.535 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:25:28.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:25:28.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:25:32.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:25:32.169 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:25:32.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:25:32.172 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:25:32.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:25:32.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2023-06-09T05:25:58.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:25:58.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:26:28.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:26:28.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:26:33.607 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:26:33.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:26:59.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:26:59.079 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:27:29.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:27:29.157 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:27:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:27:35.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:27:35.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:27:35.052 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:27:35.052 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:27:35.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2023-06-09T05:27:59.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:27:59.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:28:29.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:28:29.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:28:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:28:36.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:28:59.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:28:59.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:29:29.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:29:29.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:29:37.927 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:29:37.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:29:37.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:29:37.932 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:29:37.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:29:37.932 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2023-06-09T05:29:59.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:29:59.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:30:29.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:30:29.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:30:39.367 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:30:39.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:30:59.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:30:59.842 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:31:29.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:31:29.967 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:31:40.807 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:31:40.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:31:40.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:31:40.812 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:31:40.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:31:40.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2023-06-09T05:32:00.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:32:00.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:32:30.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:32:30.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:32:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:32:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:33:00.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:33:00.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:33:30.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:33:30.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:33:43.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:33:43.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:33:43.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:33:43.692 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:33:43.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:33:43.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2023-06-09T05:34:00.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:34:00.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:34:30.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:34:30.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:34:45.127 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:34:45.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:35:00.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:35:00.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:35:31.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:35:31.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:35:46.567 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:35:46.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:35:46.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:35:46.572 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:35:46.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:35:46.572 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2023-06-09T05:36:01.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:36:01.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:36:31.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:36:31.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:36:48.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:36:48.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:37:01.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:37:01.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:37:31.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:37:31.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:37:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:37:49.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:37:49.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:37:49.452 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:37:49.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:37:49.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2023-06-09T05:38:02.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:38:02.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:38:32.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:38:32.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:38:50.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:38:50.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:39:02.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:39:02.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:39:32.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:39:32.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:39:52.327 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:39:52.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:39:52.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:39:52.332 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:39:52.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:39:52.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2023-06-09T05:40:02.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:40:02.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:40:32.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:40:32.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:40:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:40:53.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:41:03.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:41:03.066 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:41:33.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:41:33.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:41:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:41:55.209 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:41:55.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:41:55.212 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:41:55.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:41:55.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2023-06-09T05:42:03.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:42:03.377 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:42:33.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:42:33.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:42:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:42:56.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:43:03.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:43:03.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:43:33.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:43:33.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:43:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:43:58.089 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:43:58.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:43:58.092 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:43:58.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:43:58.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2023-06-09T05:44:03.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:44:03.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:44:34.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:44:34.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:44:59.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:44:59.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:45:04.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:45:04.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:45:34.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:45:34.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:46:00.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:46:00.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:46:00.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:46:00.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:46:00.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:46:00.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2023-06-09T05:46:04.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:46:04.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:46:34.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:46:34.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:47:02.407 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:47:02.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:47:04.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:47:04.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:47:35.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:47:35.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:48:03.871 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:48:03.872 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:48:03.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:48:03.875 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:48:03.875 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:48:03.876 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2023-06-09T05:48:05.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:48:05.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:48:35.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:48:35.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:49:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:49:05.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:49:05.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:49:05.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:49:35.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:49:35.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:50:05.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:50:05.873 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:50:06.727 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:50:06.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:50:06.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:50:06.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:50:06.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:50:06.732 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2023-06-09T05:50:36.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:50:36.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:51:06.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:51:06.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:51:08.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:51:08.169 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:51:36.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:51:36.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:52:06.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:52:06.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:52:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:52:09.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:52:09.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:52:09.611 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:52:09.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:52:09.612 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2023-06-09T05:52:36.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:52:36.611 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:53:06.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:53:06.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:53:11.047 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:53:11.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:53:36.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:53:36.960 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:54:07.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:54:07.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:54:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:54:12.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:54:12.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:54:12.492 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:54:12.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:54:12.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2023-06-09T05:54:37.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:54:37.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:55:07.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:55:07.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:55:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:55:13.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:55:37.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:55:37.575 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:56:07.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:56:07.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:56:15.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:56:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:56:15.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:56:15.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:56:15.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:56:15.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2023-06-09T05:56:37.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:56:37.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:57:07.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:57:07.830 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:57:16.807 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:57:16.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:57:37.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:57:37.999 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:58:08.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:58:08.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:58:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:58:18.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T05:58:18.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T05:58:18.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T05:58:18.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T05:58:18.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2023-06-09T05:58:38.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:58:38.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:59:08.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:59:08.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:59:19.687 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T05:59:19.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T05:59:38.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T05:59:38.737 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:00:08.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:00:08.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:00:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:00:21.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:00:21.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:00:21.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:00:21.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:00:21.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2023-06-09T06:00:39.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:00:39.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:01:09.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:01:09.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:01:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:01:22.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:01:39.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:01:39.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:02:09.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:02:09.420 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:02:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:02:24.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:02:24.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:02:24.011 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:02:24.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:02:24.012 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2023-06-09T06:02:39.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:02:39.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:03:09.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:03:09.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:03:25.447 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:03:25.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:03:39.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:03:39.935 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:04:10.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:04:10.079 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:04:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:04:26.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:04:26.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:04:26.912 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:04:26.912 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:04:26.912 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2023-06-09T06:04:40.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:04:40.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:05:10.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:05:10.429 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:05:28.327 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:05:28.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:05:40.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:05:40.557 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:06:10.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:06:10.626 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:06:29.767 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:06:29.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:06:29.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:06:29.772 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:06:29.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:06:29.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2023-06-09T06:06:40.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:06:40.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:07:10.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:07:10.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:07:31.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:07:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:07:40.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:07:40.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:08:11.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:08:11.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:08:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:08:32.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:08:32.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:08:32.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:08:32.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:08:32.652 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2023-06-09T06:08:41.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:08:41.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:09:11.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:09:11.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:09:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:09:34.089 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:09:41.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:09:41.309 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:10:11.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:10:11.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:10:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:10:35.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:10:35.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:10:35.532 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:10:35.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:10:35.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2023-06-09T06:10:41.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:10:41.492 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:11:11.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:11:11.562 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:11:36.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:11:36.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:11:41.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:11:41.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:12:11.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:12:11.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:12:38.407 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:12:38.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:12:38.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:12:38.412 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:12:38.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:12:38.412 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2023-06-09T06:12:41.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:12:41.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:13:12.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:13:12.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:13:39.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:13:39.856 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:13:42.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:13:42.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:14:12.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:14:12.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:14:41.287 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:14:41.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:14:41.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:14:41.292 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:14:41.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:14:41.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2023-06-09T06:14:42.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:14:42.721 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:15:12.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:15:12.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:15:42.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:15:42.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:15:43.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:15:43.127 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:16:13.184 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:16:13.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:16:43.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:16:43.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:16:44.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:16:44.169 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:16:44.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:16:44.172 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:16:44.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:16:44.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2023-06-09T06:17:13.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:17:13.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:17:43.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:17:43.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:17:45.607 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:17:45.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:18:13.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:18:13.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:18:43.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:18:43.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:18:47.047 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:18:47.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:18:47.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:18:47.052 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:18:47.052 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:18:47.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2023-06-09T06:19:13.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:19:13.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:19:43.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:19:43.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:19:48.487 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:19:48.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:20:13.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:20:13.960 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:20:44.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:20:44.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:20:49.927 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:20:49.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:20:49.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:20:49.932 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:20:49.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:20:49.933 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2023-06-09T06:21:14.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:21:14.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:21:44.220 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:21:44.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:21:51.367 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:21:51.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:22:14.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:22:14.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:22:44.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:22:44.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:22:52.807 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:22:52.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:22:52.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:22:52.812 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:22:52.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:22:52.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2023-06-09T06:23:14.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:23:14.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:23:44.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:23:44.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:23:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:23:54.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:24:14.859 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:24:14.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:24:44.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:24:44.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:24:55.687 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:24:55.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:24:55.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:24:55.692 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:24:55.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:24:55.692 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2023-06-09T06:25:15.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:25:15.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:25:45.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:25:45.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:25:57.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:25:57.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:26:15.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:26:15.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:26:45.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:26:45.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:26:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:26:58.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:26:58.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:26:58.572 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:26:58.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:26:58.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2023-06-09T06:27:15.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:27:15.642 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:27:45.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:27:45.714 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:28:00.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:28:00.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:28:15.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:28:15.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:28:45.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:28:45.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:29:01.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:29:01.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:29:01.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:29:01.452 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:29:01.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:29:01.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2023-06-09T06:29:15.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:29:15.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:29:46.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:29:46.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:30:02.888 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:30:02.890 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:30:16.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:30:16.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:30:46.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:30:46.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:31:04.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:31:04.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:31:04.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:31:04.332 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:31:04.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:31:04.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2023-06-09T06:31:16.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:31:16.391 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:31:46.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:31:46.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:32:05.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:32:05.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:32:16.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:32:16.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:32:46.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:32:46.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:33:07.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:33:07.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:33:07.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:33:07.212 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:33:07.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:33:07.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2023-06-09T06:33:16.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:33:16.983 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:33:47.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:33:47.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:34:08.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:34:08.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:34:17.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:34:17.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:34:47.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:34:47.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:35:10.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:35:10.089 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:35:10.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:35:10.092 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:35:10.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:35:10.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2023-06-09T06:35:17.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:35:17.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:35:47.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:35:47.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:36:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:36:11.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:36:17.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:36:17.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:36:48.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:36:48.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:37:12.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:37:12.969 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:37:12.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:37:12.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:37:12.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:37:12.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2023-06-09T06:37:18.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:37:18.157 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:37:48.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:37:48.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:38:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:38:14.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:38:18.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:38:18.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:38:48.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:38:48.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:39:15.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:39:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:39:15.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:39:15.860 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:39:15.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:39:15.861 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2023-06-09T06:39:18.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:39:18.707 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:39:49.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:39:49.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:40:17.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:40:17.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:40:19.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:40:19.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:40:49.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:40:49.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:41:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:41:18.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:41:18.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:41:18.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:41:18.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:41:18.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2023-06-09T06:41:19.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:41:19.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:41:49.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:41:49.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:42:19.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:42:19.485 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:42:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:42:20.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:42:49.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:42:49.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:43:19.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:43:19.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:43:21.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:43:21.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:43:21.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:43:21.612 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:43:21.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:43:21.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2023-06-09T06:43:49.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:43:49.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:44:19.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:44:19.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:44:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:44:23.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:44:49.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:44:49.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:45:20.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:45:20.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:45:24.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:45:24.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:45:24.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:45:24.492 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:45:24.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:45:24.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2023-06-09T06:45:50.197 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:45:50.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:46:20.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:46:20.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:46:25.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:46:25.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:46:50.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:46:50.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:47:20.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:47:20.571 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:47:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:47:27.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:47:27.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:47:27.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:47:27.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:47:27.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2023-06-09T06:47:50.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:47:50.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:48:20.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:48:20.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:48:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:48:28.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:48:50.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:48:50.968 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:49:21.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:49:21.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:49:30.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:49:30.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:49:30.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:49:30.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:49:30.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:49:30.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2023-06-09T06:49:51.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:49:51.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:50:21.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:50:21.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:50:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:50:31.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:50:51.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:50:51.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:51:21.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:51:21.678 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:51:33.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:51:33.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:51:33.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:51:33.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:51:33.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:51:33.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2023-06-09T06:51:51.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:51:51.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:52:22.016 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:52:22.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:52:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:52:34.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:52:52.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:52:52.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:53:22.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:53:22.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:53:36.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:53:36.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:53:36.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:53:36.012 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:53:36.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:53:36.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2023-06-09T06:53:52.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:53:52.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:54:22.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:54:22.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:54:37.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:54:37.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:54:52.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:54:52.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:55:22.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:55:22.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:55:38.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:55:38.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:55:38.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:55:38.900 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:55:38.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:55:38.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2023-06-09T06:55:53.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:55:53.015 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:56:23.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:56:23.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:56:40.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:56:40.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:56:53.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:56:53.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:57:23.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:57:23.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:57:41.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:57:41.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:57:41.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:57:41.772 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:57:41.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:57:41.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2023-06-09T06:57:53.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:57:53.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:58:23.749 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:58:23.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:58:43.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:58:43.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T06:58:53.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:58:53.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:59:24.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:59:24.051 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:59:44.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T06:59:44.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T06:59:44.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T06:59:44.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T06:59:44.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T06:59:44.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2023-06-09T06:59:54.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T06:59:54.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:00:24.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:00:24.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:00:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:00:46.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:00:54.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:00:54.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:01:24.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:01:24.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:01:47.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:01:47.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:01:47.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:01:47.533 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:01:47.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:01:47.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2023-06-09T07:01:54.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:01:54.676 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:02:24.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:02:24.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:02:48.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:02:48.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:02:54.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:02:54.979 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:03:25.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:03:25.126 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:03:50.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:03:50.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:03:50.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:03:50.412 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:03:50.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:03:50.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2023-06-09T07:03:55.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:03:55.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:04:25.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:04:25.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:04:51.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:04:51.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:04:55.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:04:55.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:05:25.884 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:05:25.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:05:53.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:05:53.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:05:53.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:05:53.292 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:05:53.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:05:53.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2023-06-09T07:05:55.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:05:55.997 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:06:26.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:06:26.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:06:54.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:06:54.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:06:56.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:06:56.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:07:26.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:07:26.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:07:56.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:07:56.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:07:56.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:07:56.173 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:07:56.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:07:56.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2023-06-09T07:07:56.768 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:07:56.770 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:08:26.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:08:26.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:08:56.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:08:57.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:08:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:08:57.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:09:27.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:09:27.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:09:57.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:09:57.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:09:59.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:09:59.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:09:59.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:09:59.052 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:09:59.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:09:59.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2023-06-09T07:10:27.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:10:27.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:10:57.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:10:57.628 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:11:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:11:00.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:11:27.692 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:11:27.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:11:57.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:11:57.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:12:01.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:12:01.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:12:01.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:12:01.933 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:12:01.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:12:01.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2023-06-09T07:12:27.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:12:27.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:12:58.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:12:58.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:13:03.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:13:03.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:13:28.137 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:13:28.140 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:13:58.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:13:58.286 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:14:04.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:14:04.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:14:04.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:14:04.812 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:14:04.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:14:04.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2023-06-09T07:14:28.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:14:28.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:14:58.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:14:58.535 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:15:06.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:15:06.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:15:28.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:15:28.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:15:58.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:15:58.835 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:16:07.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:16:07.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:16:07.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:16:07.693 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:16:07.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:16:07.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2023-06-09T07:16:28.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:16:28.964 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:16:59.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:16:59.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:17:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:17:09.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:17:29.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:17:29.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:17:59.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:17:59.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:18:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:18:10.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:18:10.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:18:10.573 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:18:10.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:18:10.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2023-06-09T07:18:29.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:18:29.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:18:59.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:18:59.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:19:12.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:19:12.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:19:29.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:19:29.789 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:20:00.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:20:00.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:20:13.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:20:13.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:20:13.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:20:13.453 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:20:13.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:20:13.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2023-06-09T07:20:30.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:20:30.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:21:00.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:21:00.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:21:14.912 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:21:14.913 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:21:30.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:21:30.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:22:00.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:22:00.675 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:22:16.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:22:16.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:22:16.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:22:16.333 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:22:16.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:22:16.334 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2023-06-09T07:22:30.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:22:30.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:23:00.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:23:00.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:23:17.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:23:17.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:23:31.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:23:31.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:24:01.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:24:01.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:24:19.209 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:24:19.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:24:19.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:24:19.213 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:24:19.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:24:19.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2023-06-09T07:24:31.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:24:31.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:25:01.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:25:01.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:25:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:25:20.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:25:31.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:25:31.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:26:01.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:26:01.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:26:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:26:22.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:26:22.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:26:22.093 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:26:22.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:26:22.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2023-06-09T07:26:31.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:26:31.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:27:02.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:27:02.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:27:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:27:23.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:27:32.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:27:32.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:28:02.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:28:02.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:28:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:28:24.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:28:24.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:28:24.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:28:24.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:28:24.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2023-06-09T07:28:32.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:28:32.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:29:02.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:29:02.674 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:29:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:29:26.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:29:32.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:29:32.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:30:02.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:30:02.965 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:30:27.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:30:27.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:30:27.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:30:27.873 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:30:27.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:30:27.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2023-06-09T07:30:33.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:30:33.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:31:03.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:31:03.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:31:29.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:31:29.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:31:33.266 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:31:33.269 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:32:03.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:32:03.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:32:30.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:32:30.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:32:30.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:32:30.733 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:32:30.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:32:30.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2023-06-09T07:32:33.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:32:33.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:33:03.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:33:03.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:33:32.169 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:33:32.171 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:33:33.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:33:33.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:34:03.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:34:03.855 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:34:33.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:34:33.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:34:33.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:34:33.613 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:34:33.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:34:33.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2023-06-09T07:34:33.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:34:33.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:35:04.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:35:04.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:35:34.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:35:34.264 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:35:35.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:35:35.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:36:04.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:36:04.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:36:34.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:36:34.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:36:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:36:36.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:36:36.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:36:36.493 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:36:36.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:36:36.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2023-06-09T07:37:04.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:37:04.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:37:34.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:37:34.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:37:37.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:37:37.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:38:05.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:38:05.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:38:35.180 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:38:35.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:38:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:38:39.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:38:39.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:38:39.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:38:39.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:38:39.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2023-06-09T07:39:05.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:39:05.321 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:39:35.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:39:35.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:39:40.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:39:40.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:40:05.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:40:05.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:40:35.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:40:35.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:40:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:40:42.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:40:42.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:40:42.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:40:42.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:40:42.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2023-06-09T07:41:06.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:41:06.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:41:36.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:41:36.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:41:43.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:41:43.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:42:06.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:42:06.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:42:36.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:42:36.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:42:45.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:42:45.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:42:45.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:42:45.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:42:45.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:42:45.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2023-06-09T07:43:06.854 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:43:06.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:43:36.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:43:36.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:43:46.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:43:46.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:44:07.134 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:44:07.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:44:37.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:44:37.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:44:48.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:44:48.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:44:48.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:44:48.012 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:44:48.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:44:48.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2023-06-09T07:45:07.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:45:07.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:45:37.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:45:37.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:45:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:45:49.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:46:07.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:46:07.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:46:37.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:46:37.964 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:46:50.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:46:50.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:46:50.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:46:50.900 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:46:50.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:46:50.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2023-06-09T07:47:08.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:47:08.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:47:38.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:47:38.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:47:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:47:52.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:48:08.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:48:08.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:48:38.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:48:38.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:48:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:48:53.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:48:53.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:48:53.772 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:48:53.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:48:53.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2023-06-09T07:49:08.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:49:08.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:49:39.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:49:39.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:49:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:49:55.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:50:09.212 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:50:09.215 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:50:39.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:50:39.284 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:50:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:50:56.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:50:56.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:50:56.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:50:56.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:50:56.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2023-06-09T07:51:09.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:51:09.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:51:39.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:51:39.516 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:51:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:51:58.089 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:52:09.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:52:09.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:52:39.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:52:39.988 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:52:59.527 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:52:59.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:52:59.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:52:59.532 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:52:59.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:52:59.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2023-06-09T07:53:10.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:53:10.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:53:40.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:53:40.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:54:00.967 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:54:00.969 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:54:10.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:54:10.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:54:40.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:54:40.571 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:55:02.407 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:55:02.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:55:02.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:55:02.412 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:55:02.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:55:02.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2023-06-09T07:55:10.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:55:10.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:55:40.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:55:40.853 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:56:03.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:56:03.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:56:10.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:56:10.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:56:41.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:56:41.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:57:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:57:05.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:57:05.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:57:05.292 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:57:05.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:57:05.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2023-06-09T07:57:11.271 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:57:11.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:57:41.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:57:41.420 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:58:06.727 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:58:06.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T07:58:11.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:58:11.556 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:58:41.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:58:41.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:59:08.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T07:59:08.169 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T07:59:08.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T07:59:08.172 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T07:59:08.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T07:59:08.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2023-06-09T07:59:11.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:59:11.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:59:42.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T07:59:42.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:00:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:00:09.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:00:12.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:00:12.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:00:42.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:00:42.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:01:11.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:01:11.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:01:11.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:01:11.052 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:01:11.052 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:01:11.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2023-06-09T08:01:12.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:01:12.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:01:42.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:01:42.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:02:12.487 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:02:12.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:02:12.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:02:12.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:02:42.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:02:42.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:03:13.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:03:13.090 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:03:13.927 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:03:13.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:03:13.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:03:13.931 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:03:13.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:03:13.932 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2023-06-09T08:03:43.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:03:43.235 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:04:13.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:04:13.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:04:15.367 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:04:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:04:43.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:04:43.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:05:13.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:05:13.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:05:16.807 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:05:16.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:05:16.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:05:16.811 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:05:16.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:05:16.812 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2023-06-09T08:05:43.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:05:43.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:06:13.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:06:13.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:06:18.247 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:06:18.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:06:44.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:06:44.126 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:07:14.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:07:14.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:07:19.687 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:07:19.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:07:19.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:07:19.691 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:07:19.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:07:19.692 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2023-06-09T08:07:44.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:07:44.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:08:14.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:08:14.580 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:08:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:08:21.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:08:44.851 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:08:44.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:09:14.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:09:14.916 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:09:22.567 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:09:22.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:09:22.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:09:22.571 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:09:22.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:09:22.572 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2023-06-09T08:09:45.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:09:45.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:10:15.218 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:10:15.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:10:24.007 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:10:24.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:10:45.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:10:45.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:11:15.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:11:15.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:11:25.447 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:11:25.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:11:25.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:11:25.452 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:11:25.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:11:25.452 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2023-06-09T08:11:45.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:11:45.666 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:12:15.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:12:15.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:12:26.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:12:26.896 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:12:45.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:12:45.983 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:13:16.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:13:16.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:13:28.327 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:13:28.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:13:28.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:13:28.332 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:13:28.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:13:28.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2023-06-09T08:13:46.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:13:46.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:14:16.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:14:16.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:14:29.767 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:14:29.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:14:46.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:14:46.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:15:16.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:15:16.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:15:31.207 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:15:31.209 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:15:31.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:15:31.212 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:15:31.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:15:31.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2023-06-09T08:15:46.896 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:15:46.899 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:16:17.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:16:17.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:16:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:16:32.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:16:47.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:16:47.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:17:17.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:17:17.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:17:34.087 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:17:34.089 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:17:34.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:17:34.092 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:17:34.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:17:34.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2023-06-09T08:17:47.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:17:47.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:18:17.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:18:17.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:18:35.527 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:18:35.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:18:48.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:18:48.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:19:18.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:19:18.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:19:36.967 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:19:36.969 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:19:36.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:19:36.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:19:36.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:19:36.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2023-06-09T08:19:48.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:19:48.415 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:20:18.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:20:18.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:20:38.407 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:20:38.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:20:48.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:20:48.841 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:21:18.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:21:18.994 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:21:39.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:21:39.856 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:21:39.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:21:39.860 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:21:39.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:21:39.860 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2023-06-09T08:21:49.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:21:49.115 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:22:19.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:22:19.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:22:41.287 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:22:41.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:22:49.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:22:49.557 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:23:19.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:23:19.702 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:23:42.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:23:42.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:23:42.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:23:42.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:23:42.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:23:42.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2023-06-09T08:23:49.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:23:49.847 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:24:20.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:24:20.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:24:44.167 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:24:44.169 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:24:50.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:24:50.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:25:20.340 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:25:20.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:25:45.607 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:25:45.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:25:45.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:25:45.612 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:25:45.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:25:45.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2023-06-09T08:25:50.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:25:50.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:26:20.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:26:20.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:26:47.047 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:26:47.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:26:50.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:26:50.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:27:20.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:27:20.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:27:48.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:27:48.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:27:48.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:27:48.492 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:27:48.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:27:48.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2023-06-09T08:27:51.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:27:51.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:28:21.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:28:21.166 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:28:49.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:28:49.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:28:51.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:28:51.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:29:21.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:29:21.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:29:51.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:29:51.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:29:51.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:29:51.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:29:51.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:29:51.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2023-06-09T08:29:51.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:29:51.553 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:30:21.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:30:21.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:30:51.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:30:51.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:30:52.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:30:52.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:31:21.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:31:21.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:31:52.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:31:52.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:31:54.247 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:31:54.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:31:54.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:31:54.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:31:54.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:31:54.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2023-06-09T08:32:22.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:32:22.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:32:52.391 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:32:52.394 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:32:55.687 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:32:55.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:33:22.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:33:22.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:33:52.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:33:52.710 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:33:57.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:33:57.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:33:57.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:33:57.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:33:57.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:33:57.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2023-06-09T08:34:22.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:34:22.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:34:53.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:34:53.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:34:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:34:58.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:35:23.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:35:23.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:35:53.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:35:53.293 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:36:00.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:36:00.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:36:00.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:36:00.012 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:36:00.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:36:00.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2023-06-09T08:36:23.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:36:23.372 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:36:53.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:36:53.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:37:01.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:37:01.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:37:23.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:37:23.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:37:53.636 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:37:53.639 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:38:02.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:38:02.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:38:02.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:38:02.900 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:38:02.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:38:02.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2023-06-09T08:38:23.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:38:23.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:38:53.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:38:53.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:39:04.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:39:04.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:39:24.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:39:24.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:39:54.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:39:54.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:40:05.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:40:05.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:40:05.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:40:05.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:40:05.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:40:05.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2023-06-09T08:40:24.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:40:24.479 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:40:54.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:40:54.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:41:07.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:41:07.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:41:24.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:41:24.763 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:41:54.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:41:54.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:42:08.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:42:08.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:42:08.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:42:08.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:42:08.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:42:08.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2023-06-09T08:42:25.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:42:25.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:42:55.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:42:55.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:43:10.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:43:10.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:43:25.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:43:25.504 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:43:55.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:43:55.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:44:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:44:11.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:44:11.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:44:11.532 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:44:11.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:44:11.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2023-06-09T08:44:25.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:44:25.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:44:56.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:44:56.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:45:12.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:45:12.969 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:45:26.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:45:26.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:45:56.420 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:45:56.423 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:46:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:46:14.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:46:14.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:46:14.412 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:46:14.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:46:14.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2023-06-09T08:46:26.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:46:26.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:46:56.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:46:56.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:47:15.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:47:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:47:26.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:47:26.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:47:57.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:47:57.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:48:17.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:48:17.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:48:17.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:48:17.292 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:48:17.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:48:17.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2023-06-09T08:48:27.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:48:27.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:48:57.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:48:57.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:49:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:49:18.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:49:27.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:49:27.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:49:57.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:49:57.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:50:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:50:20.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:50:20.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:50:20.173 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:50:20.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:50:20.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2023-06-09T08:50:27.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:50:27.902 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:50:58.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:50:58.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:51:21.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:51:21.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:51:28.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:51:28.345 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:51:58.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:51:58.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:52:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:52:23.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:52:23.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:52:23.052 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:52:23.052 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:52:23.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2023-06-09T08:52:28.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:52:28.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:52:58.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:52:58.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:53:24.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:53:24.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:53:28.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:53:28.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:53:59.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:53:59.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:54:25.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:54:25.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:54:25.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:54:25.932 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:54:25.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:54:25.933 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2023-06-09T08:54:29.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:54:29.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:54:59.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:54:59.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:55:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:55:27.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:55:29.498 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:55:29.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:55:59.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:55:59.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:56:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:56:28.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:56:28.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:56:28.812 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:56:28.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:56:28.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2023-06-09T08:56:29.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:56:29.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:56:59.943 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:56:59.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:57:30.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:57:30.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:57:30.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:57:30.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T08:58:00.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:58:00.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:58:30.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:58:30.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:58:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:58:31.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T08:58:31.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T08:58:31.693 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T08:58:31.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T08:58:31.694 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2023-06-09T08:59:00.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:59:00.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:59:30.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:59:30.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T08:59:33.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T08:59:33.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:00:00.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:00:00.762 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:00:30.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:00:30.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:00:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:00:34.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:00:34.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:00:34.572 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:00:34.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:00:34.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2023-06-09T09:01:01.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:01:01.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:01:31.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:01:31.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:01:36.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:01:36.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:02:01.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:02:01.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:02:31.688 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:02:31.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:02:37.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:02:37.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:02:37.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:02:37.452 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:02:37.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:02:37.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2023-06-09T09:03:01.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:03:01.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:03:31.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:03:31.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:03:38.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:03:38.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:04:02.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:04:02.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:04:32.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:04:32.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:04:40.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:04:40.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:04:40.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:04:40.332 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:04:40.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:04:40.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2023-06-09T09:05:02.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:05:02.427 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:05:32.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:05:32.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:05:41.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:05:41.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:06:02.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:06:02.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:06:32.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:06:32.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:06:43.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:06:43.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:06:43.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:06:43.212 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:06:43.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:06:43.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2023-06-09T09:07:02.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:07:02.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:07:32.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:07:32.886 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:07:44.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:07:44.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:08:02.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:08:02.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:08:33.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:08:33.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:08:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:08:46.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:08:46.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:08:46.093 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:08:46.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:08:46.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2023-06-09T09:09:03.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:09:03.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:09:33.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:09:33.331 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:09:47.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:09:47.529 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:10:03.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:10:03.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:10:33.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:10:33.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:10:48.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:10:48.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:10:48.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:10:48.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:10:48.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:10:48.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2023-06-09T09:11:03.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:11:03.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:11:34.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:11:34.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:11:50.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:11:50.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:12:04.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:12:04.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:12:34.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:12:34.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:12:51.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:12:51.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:12:51.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:12:51.860 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:12:51.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:12:51.861 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2023-06-09T09:13:04.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:13:04.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:13:34.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:13:34.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:13:53.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:13:53.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:14:04.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:14:04.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:14:35.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:14:35.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:14:54.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:14:54.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:14:54.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:14:54.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:14:54.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:14:54.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2023-06-09T09:15:05.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:15:05.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:15:35.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:15:35.327 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:15:56.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:15:56.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:16:05.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:16:05.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:16:35.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:16:35.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:16:57.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:16:57.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:16:57.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:16:57.612 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:16:57.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:16:57.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2023-06-09T09:17:05.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:17:05.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:17:36.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:17:36.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:17:59.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:17:59.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:18:06.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:18:06.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:18:36.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:18:36.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:19:00.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:19:00.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:19:00.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:19:00.492 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:19:00.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:19:00.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2023-06-09T09:19:06.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:19:06.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:19:36.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:19:36.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:20:01.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:20:01.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:20:06.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:20:06.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:20:36.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:20:36.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:21:03.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:21:03.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:21:03.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:21:03.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:21:03.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:21:03.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2023-06-09T09:21:07.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:21:07.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:21:37.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:21:37.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:22:04.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:22:04.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:22:07.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:22:07.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:22:37.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:22:37.528 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:23:06.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:23:06.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:23:06.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:23:06.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:23:06.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:23:06.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2023-06-09T09:23:07.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:23:07.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:23:37.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:23:37.979 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:24:07.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:24:07.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:24:08.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:24:08.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:24:38.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:24:38.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:25:08.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:25:08.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:25:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:25:09.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:25:09.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:25:09.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:25:09.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:25:09.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2023-06-09T09:25:38.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:25:38.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:26:08.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:26:08.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:26:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:26:10.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:26:38.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:26:38.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:27:08.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:27:08.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:27:12.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:27:12.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:27:12.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:27:12.012 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:27:12.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:27:12.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2023-06-09T09:27:38.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:27:38.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:28:09.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:28:09.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:28:13.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:28:13.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:28:39.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:28:39.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:29:09.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:29:09.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:29:14.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:29:14.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:29:14.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:29:14.900 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:29:14.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:29:14.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2023-06-09T09:29:39.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:29:39.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:30:09.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:30:09.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:30:16.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:30:16.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:30:39.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:30:39.411 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:31:09.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:31:09.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:31:17.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:31:17.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:31:17.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:31:17.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:31:17.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:31:17.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2023-06-09T09:31:39.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:31:39.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:32:09.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:32:09.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:32:19.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:32:19.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:32:39.691 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:32:39.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:33:09.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:33:09.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:33:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:33:20.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:33:20.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:33:20.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:33:20.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:33:20.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2023-06-09T09:33:39.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:33:39.835 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:34:10.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:34:10.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:34:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:34:22.089 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:34:40.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:34:40.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:35:10.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:35:10.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:35:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:35:23.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:35:23.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:35:23.533 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:35:23.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:35:23.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2023-06-09T09:35:40.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:35:40.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:36:10.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:36:10.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:36:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:36:24.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:36:40.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:36:40.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:37:10.828 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:37:10.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:37:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:37:26.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:37:26.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:37:26.412 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:37:26.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:37:26.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2023-06-09T09:37:40.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:37:40.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:38:11.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:38:11.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:38:27.848 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:38:27.850 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:38:41.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:38:41.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:39:11.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:39:11.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:39:29.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:39:29.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:39:29.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:39:29.292 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:39:29.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:39:29.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2023-06-09T09:39:41.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:39:41.676 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:40:11.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:40:11.936 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:40:30.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:40:30.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:40:42.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:40:42.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:41:12.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:41:12.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:41:32.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:41:32.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:41:32.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:41:32.172 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:41:32.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:41:32.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2023-06-09T09:41:42.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:41:42.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:42:12.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:42:12.479 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:42:33.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:42:33.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:42:42.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:42:42.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:43:12.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:43:12.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:43:35.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:43:35.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:43:35.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:43:35.052 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:43:35.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:43:35.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2023-06-09T09:43:42.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:43:42.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:44:13.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:44:13.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:44:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:44:36.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:44:43.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:44:43.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:45:13.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:45:13.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:45:37.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:45:37.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:45:37.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:45:37.932 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:45:37.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:45:37.932 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2023-06-09T09:45:43.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:45:43.550 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:46:13.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:46:13.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:46:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:46:39.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:46:43.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:46:43.824 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:47:13.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:47:13.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:47:40.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:47:40.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:47:40.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:47:40.812 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:47:40.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:47:40.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2023-06-09T09:47:44.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:47:44.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:48:14.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:48:14.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:48:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:48:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:48:44.358 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:48:44.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:49:14.491 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:49:14.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:49:43.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:49:43.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:49:43.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:49:43.692 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:49:43.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:49:43.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2023-06-09T09:49:44.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:49:44.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:50:14.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:50:14.802 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:50:44.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:50:44.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:50:45.127 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:50:45.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:51:15.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:51:15.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:51:45.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:51:45.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:51:46.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:51:46.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:51:46.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:51:46.572 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:51:46.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:51:46.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2023-06-09T09:52:15.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:52:15.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:52:45.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:52:45.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:52:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:52:48.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:53:15.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:53:15.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:53:45.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:53:45.731 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:53:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:53:49.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:53:49.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:53:49.452 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:53:49.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:53:49.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2023-06-09T09:54:15.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:54:15.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:54:46.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:54:46.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:54:50.894 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:54:50.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:55:16.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:55:16.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:55:46.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:55:46.315 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:55:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:55:52.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:55:52.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:55:52.333 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:55:52.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:55:52.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2023-06-09T09:56:16.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:56:16.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:56:46.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:56:46.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:56:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:56:53.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:57:16.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:57:16.732 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:57:46.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:57:46.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:57:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:57:55.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:57:55.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:57:55.212 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:57:55.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:57:55.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2023-06-09T09:58:17.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:58:17.011 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:58:47.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:58:47.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:58:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:58:56.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T09:59:17.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:59:17.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:59:47.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:59:47.449 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T09:59:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T09:59:58.089 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T09:59:58.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T09:59:58.092 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T09:59:58.092 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T09:59:58.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2023-06-09T10:00:17.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:00:17.585 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:00:47.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:00:47.707 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:00:59.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:00:59.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:01:18.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:01:18.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:01:48.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:01:48.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:02:00.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:02:00.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:02:00.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:02:00.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:02:00.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:02:00.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2023-06-09T10:02:18.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:02:18.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:02:48.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:02:48.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:03:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:03:02.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:03:18.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:03:18.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:03:48.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:03:48.821 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:04:03.848 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:04:03.850 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:04:03.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:04:03.853 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:04:03.853 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:04:03.853 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2023-06-09T10:04:18.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:04:18.999 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:04:49.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:04:49.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:05:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:05:05.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:05:19.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:05:19.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:05:49.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:05:49.418 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:06:06.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:06:06.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:06:06.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:06:06.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:06:06.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:06:06.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2023-06-09T10:06:19.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:06:19.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:06:49.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:06:49.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:07:08.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:07:08.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:07:19.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:07:19.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:07:49.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:07:49.999 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:08:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:08:09.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:08:09.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:08:09.612 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:08:09.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:08:09.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2023-06-09T10:08:20.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:08:20.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:08:50.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:08:50.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:09:11.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:09:11.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:09:20.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:09:20.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:09:50.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:09:50.814 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:10:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:10:12.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:10:12.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:10:12.493 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:10:12.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:10:12.494 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2023-06-09T10:10:20.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:10:20.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:10:50.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:10:50.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:11:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:11:13.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:11:21.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:11:21.015 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:11:51.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:11:51.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:12:15.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:12:15.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:12:15.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:12:15.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:12:15.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:12:15.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2023-06-09T10:12:21.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:12:21.173 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:12:51.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:12:51.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:13:16.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:13:16.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:13:21.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:13:21.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:13:51.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:13:51.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:14:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:14:18.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:14:18.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:14:18.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:14:18.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:14:18.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2023-06-09T10:14:21.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:14:21.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:14:51.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:14:51.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:15:19.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:15:19.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:15:21.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:15:21.904 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:15:51.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:15:51.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:16:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:16:21.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:16:21.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:16:21.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:16:21.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:16:21.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2023-06-09T10:16:22.099 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:16:22.101 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:16:52.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:16:52.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:17:22.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:17:22.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:17:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:17:22.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:17:52.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:17:52.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:18:22.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:18:22.381 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:18:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:18:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:18:24.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:18:24.013 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:18:24.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:18:24.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2023-06-09T10:18:52.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:18:52.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:19:22.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:19:22.507 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:19:25.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:19:25.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:19:52.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:19:52.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:20:22.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:20:22.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:20:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:20:26.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:20:26.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:20:26.900 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:20:26.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:20:26.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2023-06-09T10:20:52.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:20:52.768 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:21:22.893 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:21:22.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:21:28.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:21:28.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:21:53.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:21:53.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:22:23.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:22:23.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:22:29.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:22:29.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:22:29.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:22:29.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:22:29.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:22:29.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2023-06-09T10:22:53.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:22:53.168 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:23:23.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:23:23.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:23:31.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:23:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:23:53.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:23:53.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:24:23.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:24:23.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:24:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:24:32.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:24:32.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:24:32.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:24:32.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:24:32.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2023-06-09T10:24:53.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:24:53.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:25:23.957 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:25:23.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:25:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:25:34.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:25:54.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:25:54.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:26:24.215 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:26:24.218 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:26:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:26:35.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:26:35.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:26:35.532 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:26:35.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:26:35.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2023-06-09T10:26:54.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:26:54.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:27:24.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:27:24.368 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:27:36.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:27:36.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:27:54.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:27:54.426 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:28:24.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:28:24.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:28:38.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:28:38.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:28:38.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:28:38.413 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:28:38.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:28:38.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2023-06-09T10:28:54.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:28:54.609 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:29:24.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:29:24.787 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:29:39.850 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:29:39.852 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:29:54.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:29:54.866 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:30:24.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:30:24.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:30:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:30:41.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:30:41.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:30:41.293 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:30:41.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:30:41.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2023-06-09T10:30:55.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:30:55.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:31:25.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:31:25.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:31:42.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:31:42.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:31:55.351 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:31:55.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:32:25.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:32:25.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:32:44.169 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:32:44.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:32:44.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:32:44.173 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:32:44.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:32:44.174 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2023-06-09T10:32:55.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:32:55.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:33:25.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:33:25.640 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:33:45.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:33:45.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:33:55.697 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:33:55.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:34:25.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:34:25.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:34:47.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:34:47.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:34:47.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:34:47.053 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:34:47.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:34:47.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2023-06-09T10:34:55.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:34:55.865 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:35:26.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:35:26.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:35:48.489 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:35:48.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:35:56.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:35:56.115 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:36:26.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:36:26.252 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:36:49.929 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:36:49.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:36:49.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:36:49.933 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:36:49.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:36:49.934 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2023-06-09T10:36:56.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:36:56.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:37:26.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:37:26.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:37:51.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:37:51.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:37:56.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:37:56.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:38:26.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:38:26.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:38:52.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:38:52.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:38:52.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:38:52.813 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:38:52.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:38:52.814 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2023-06-09T10:38:56.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:38:56.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:39:27.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:39:27.103 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:39:54.249 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:39:54.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:39:57.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:39:57.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:40:27.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:40:27.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:40:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:40:55.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:40:55.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:40:55.693 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:40:55.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:40:55.694 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2023-06-09T10:40:57.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:40:57.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:41:27.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:41:27.789 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:41:57.129 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:41:57.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:41:57.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:41:57.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:42:28.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:42:28.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:42:58.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:42:58.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:42:58.569 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:42:58.571 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:42:58.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:42:58.574 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:42:58.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:42:58.575 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2023-06-09T10:43:28.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:43:28.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:43:58.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:43:58.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:44:00.009 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:44:00.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:44:28.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:44:28.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:44:58.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:44:58.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:45:01.449 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:45:01.451 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:45:01.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:45:01.453 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:45:01.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:45:01.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2023-06-09T10:45:28.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:45:28.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:45:59.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:45:59.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:46:02.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:46:02.898 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:46:29.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:46:29.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:46:59.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:46:59.373 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:47:04.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:47:04.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:47:04.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:47:04.333 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:47:04.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:47:04.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2023-06-09T10:47:29.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:47:29.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:47:59.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:47:59.689 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:48:05.769 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:48:05.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:48:29.831 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:48:29.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:48:59.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:48:59.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:49:07.209 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:49:07.211 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:49:07.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:49:07.214 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:49:07.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:49:07.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2023-06-09T10:49:30.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:49:30.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:50:00.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:50:00.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:50:08.649 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:50:08.651 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:50:30.326 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:50:30.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:51:00.468 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:51:00.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:51:10.089 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:51:10.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:51:10.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:51:10.093 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:51:10.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:51:10.094 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2023-06-09T10:51:30.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:51:30.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:52:00.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:52:00.731 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:52:11.529 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:52:11.531 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:52:30.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:52:30.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:53:01.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:53:01.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:53:12.969 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:53:12.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:53:12.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:53:12.973 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:53:12.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:53:12.974 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2023-06-09T10:53:31.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:53:31.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:54:01.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:54:01.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:54:14.409 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:54:14.411 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:54:31.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:54:31.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:55:01.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:55:01.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:55:15.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:55:15.858 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:55:15.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:55:15.861 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:55:15.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:55:15.862 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2023-06-09T10:55:31.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:55:31.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:56:02.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:56:02.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:56:17.289 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:56:17.291 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:56:32.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:56:32.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:57:02.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:57:02.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:57:18.729 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:57:18.731 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:57:18.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:57:18.734 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:57:18.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:57:18.737 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2023-06-09T10:57:32.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:57:32.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:58:02.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:58:02.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:58:20.169 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:58:20.171 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T10:58:32.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:58:32.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:59:02.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:59:02.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:59:21.609 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T10:59:21.611 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T10:59:21.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:59:21.614 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T10:59:21.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T10:59:21.616 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2023-06-09T10:59:33.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T10:59:33.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:00:03.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:00:03.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:00:23.049 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:00:23.051 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:00:33.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:00:33.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:01:03.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:01:03.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:01:24.489 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:01:24.491 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:01:24.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:01:24.496 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:01:24.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:01:24.498 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2023-06-09T11:01:33.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:01:33.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:02:03.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:02:03.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:02:25.930 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:02:25.931 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:02:33.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:02:33.893 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:03:04.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:03:04.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:03:27.369 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:03:27.371 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:03:27.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:03:27.373 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:03:27.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:03:27.374 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2023-06-09T11:03:34.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:03:34.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:04:04.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:04:04.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:04:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:04:28.811 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:04:34.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:04:34.546 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:05:04.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:05:04.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:05:30.250 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:05:30.251 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:05:30.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:05:30.255 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:05:30.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:05:30.256 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2023-06-09T11:05:34.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:05:34.814 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:06:04.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:06:04.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:06:31.690 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:06:31.691 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:06:35.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:06:35.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:07:05.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:07:05.202 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:07:33.129 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:07:33.131 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:07:33.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:07:33.134 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:07:33.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:07:33.134 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2023-06-09T11:07:35.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:07:35.372 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:08:05.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:08:05.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:08:34.569 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:08:34.571 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:08:35.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:08:35.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:05.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:05.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:36.009 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:09:36.011 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:09:36.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:09:36.014 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:09:36.014 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:09:36.015 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2023-06-09T11:09:36.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:36.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:06.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:06.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:36.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:36.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:37.452 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:10:37.453 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:11:06.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:06.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:36.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:36.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:38.896 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:11:38.900 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:11:38.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:11:38.907 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:11:38.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:11:38.908 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2023-06-09T11:12:06.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:06.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:37.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:37.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:40.329 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:12:40.332 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:13:07.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:07.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:37.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:37.283 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:41.769 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:13:41.771 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:13:41.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:13:41.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:13:41.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:13:41.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2023-06-09T11:14:07.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:07.621 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:37.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:37.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:43.210 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:14:43.211 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:15:07.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:07.878 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:37.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:37.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:44.649 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:15:44.651 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:15:44.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:15:44.653 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:15:44.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:15:44.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2023-06-09T11:16:08.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:08.049 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:38.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:38.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:46.090 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:16:46.091 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:17:08.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:08.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:38.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:38.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:47.529 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:17:47.531 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:17:47.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:17:47.534 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:17:47.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:17:47.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2023-06-09T11:18:08.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:18:08.331 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:18:38.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:18:38.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:18:48.970 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:18:48.971 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:19:08.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:08.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:38.564 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:38.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:50.409 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:19:50.411 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:19:50.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:19:50.414 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:19:50.414 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:19:50.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2023-06-09T11:20:08.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:20:08.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:20:38.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:20:38.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:20:51.873 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:20:51.874 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:21:09.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:09.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:39.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:39.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:21:53.291 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:21:53.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:21:53.294 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:21:53.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:21:53.295 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2023-06-09T11:22:09.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:22:09.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:22:39.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:22:39.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:22:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:22:54.731 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:23:09.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:09.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:39.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:39.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:56.170 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:23:56.171 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:23:56.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:23:56.174 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:23:56.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:23:56.175 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2023-06-09T11:24:09.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:09.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:39.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:40.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:57.609 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:24:57.611 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:25:10.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:25:10.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:25:40.255 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:25:40.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:25:59.050 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:25:59.051 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:25:59.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:25:59.054 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:25:59.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:25:59.055 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2023-06-09T11:26:10.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:26:10.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:26:40.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:26:40.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:00.490 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:27:00.492 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:27:10.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:10.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:40.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:40.692 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:28:01.930 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:28:01.931 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:28:01.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:28:01.934 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:28:01.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:28:01.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2023-06-09T11:28:10.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:28:10.754 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:28:40.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:28:40.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:03.370 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:29:03.372 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:29:10.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:10.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:41.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:41.018 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:04.810 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:30:04.811 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:30:04.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:30:04.814 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:30:04.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:30:04.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2023-06-09T11:30:11.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:11.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:41.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:41.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:31:06.250 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:31:06.252 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:31:11.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:31:11.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:31:41.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:31:41.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:07.690 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:32:07.692 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:32:07.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:32:07.695 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:32:07.695 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:32:07.696 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2023-06-09T11:32:11.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:11.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:41.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:41.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:09.130 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:33:09.132 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:33:11.760 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:11.763 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:41.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:41.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:10.570 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:34:10.571 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:34:10.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:34:10.574 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:34:10.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:34:10.575 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2023-06-09T11:34:12.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:12.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:42.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:42.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:35:12.010 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:35:12.012 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:35:12.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:35:12.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:35:42.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:35:42.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:12.634 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:12.637 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:13.450 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:36:13.452 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:36:13.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:36:13.454 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:36:13.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:36:13.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2023-06-09T11:36:42.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:42.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:37:12.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:37:12.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:37:14.896 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:37:14.899 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:37:43.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:37:43.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:13.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:13.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:16.329 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:38:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:38:16.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:38:16.334 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:38:16.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:38:16.335 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2023-06-09T11:38:43.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:43.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:39:13.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:39:13.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:39:17.769 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:39:17.771 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:39:43.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:39:43.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:13.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:13.802 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:19.209 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:40:19.211 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:40:19.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:40:19.214 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:40:19.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:40:19.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2023-06-09T11:40:43.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:43.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:14.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:14.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:20.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:41:20.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:41:44.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:44.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:14.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:14.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:42:22.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:42:22.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:42:22.093 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:42:22.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:42:22.094 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2023-06-09T11:42:44.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:44.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:14.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:14.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:23.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:43:23.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:43:44.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:44.844 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:14.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:14.976 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:24.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:44:24.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:44:24.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:44:24.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:44:24.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:44:24.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2023-06-09T11:44:45.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:45.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:15.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:15.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:45:26.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:45:45.570 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:45.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:15.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:15.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:27.854 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:46:27.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:46:27.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:46:27.860 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:46:27.860 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:46:27.861 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2023-06-09T11:46:45.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:45.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:15.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:15.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:29.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:47:29.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:47:46.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:46.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:48:16.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:48:16.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:48:30.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:48:30.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:48:30.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:48:30.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:48:30.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:48:30.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2023-06-09T11:48:46.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:48:46.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:16.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:16.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:32.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:49:32.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:49:46.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:46.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:16.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:16.789 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:33.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:50:33.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:50:33.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:50:33.612 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:50:33.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:50:33.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2023-06-09T11:50:46.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:46.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:16.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:16.997 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:51:35.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:51:47.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:47.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:17.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:17.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:36.487 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:52:36.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:52:36.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:52:36.492 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:52:36.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:52:36.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2023-06-09T11:52:47.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:47.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:53:17.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:53:17.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:53:37.927 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:53:37.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:53:47.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:53:47.775 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:17.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:17.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:39.367 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:54:39.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:54:39.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:54:39.371 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:54:39.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:54:39.372 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2023-06-09T11:54:48.039 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:48.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:55:18.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:55:18.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:55:40.807 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:55:40.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:55:48.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:55:48.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:18.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:18.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:56:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:56:42.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:56:42.252 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:56:42.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:56:42.252 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2023-06-09T11:56:48.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:48.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:18.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:18.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:43.687 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:57:43.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:57:48.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:48.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:18.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:18.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:45.127 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:58:45.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T11:58:45.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:58:45.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T11:58:45.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T11:58:45.132 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2023-06-09T11:58:49.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:49.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:59:19.258 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:59:19.261 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:59:46.567 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T11:59:46.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T11:59:49.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:59:49.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:19.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:19.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:00:48.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:00:48.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:00:48.012 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:00:48.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:00:48.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2023-06-09T12:00:49.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:49.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:19.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:19.846 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:01:49.449 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:01:50.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:50.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:20.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:20.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:50.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:50.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:50.889 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:02:50.892 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:02:50.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:02:50.905 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:02:50.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:02:50.906 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2023-06-09T12:03:20.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:20.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:50.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:50.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:52.327 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:03:52.329 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:04:20.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:20.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:50.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:50.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:53.767 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:04:53.769 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:04:53.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:04:53.771 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:04:53.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:04:53.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2023-06-09T12:05:21.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:21.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:51.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:51.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:55.207 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:05:55.209 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:06:21.357 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:21.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:51.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:51.497 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:56.647 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:06:56.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:06:56.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:06:56.652 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:06:56.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:06:56.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2023-06-09T12:07:21.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:21.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:51.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:51.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:58.087 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:07:58.088 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:08:21.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:21.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:52.085 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:52.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:59.527 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:08:59.528 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:08:59.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:08:59.531 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:08:59.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:08:59.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2023-06-09T12:09:22.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:09:22.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:09:52.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:09:52.415 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:00.967 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:10:00.969 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:10:22.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:22.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:52.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:52.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:02.407 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:11:02.409 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:11:02.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:11:02.411 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:11:02.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:11:02.412 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2023-06-09T12:11:22.989 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:22.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:53.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:53.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:03.853 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:12:03.856 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:12:23.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:23.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:53.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:53.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:05.287 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:13:05.289 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:13:05.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:13:05.291 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:13:05.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:13:05.292 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2023-06-09T12:13:23.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:23.782 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:53.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:53.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:06.727 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:14:06.729 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:14:24.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:24.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:54.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:54.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:15:08.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:15:08.169 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:15:08.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:15:08.172 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:15:08.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:15:08.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2023-06-09T12:15:24.337 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:15:24.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:15:54.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:15:54.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:16:09.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:16:24.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:24.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:54.783 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:54.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:11.047 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:17:11.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:17:11.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:17:11.052 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:17:11.052 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:17:11.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2023-06-09T12:17:24.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:24.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:55.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:55.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:18:12.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:18:25.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:25.348 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:55.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:55.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:19:13.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:19:13.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:19:13.932 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:19:13.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:19:13.933 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2023-06-09T12:19:25.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:25.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:55.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:55.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:20:15.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:20:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:20:25.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:20:25.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:20:55.840 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:20:55.843 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:21:16.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:21:16.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:21:16.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:21:16.812 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:21:16.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:21:16.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2023-06-09T12:21:25.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:21:25.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:21:55.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:21:55.968 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:22:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:22:18.249 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:22:26.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:22:26.070 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:22:56.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:22:56.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:23:19.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:23:19.689 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:23:19.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:23:19.692 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:23:19.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:23:19.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2023-06-09T12:23:26.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:23:26.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:23:56.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:23:56.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:24:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:24:21.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:24:26.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:24:26.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:24:56.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:24:56.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:25:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:25:22.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:25:22.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:25:22.573 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:25:22.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:25:22.574 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2023-06-09T12:25:26.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:25:26.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:25:57.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:25:57.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:26:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:26:24.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:26:27.176 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:26:27.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:26:57.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:26:57.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:27:25.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:27:25.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:27:25.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:27:25.453 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:27:25.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:27:25.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2023-06-09T12:27:27.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:27:27.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:27:57.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:27:57.559 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:28:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:28:26.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:28:27.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:28:27.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:28:57.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:28:57.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:29:27.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:29:27.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:29:28.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:29:28.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:29:28.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:29:28.332 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:29:28.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:29:28.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2023-06-09T12:29:58.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:29:58.006 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:30:28.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:30:28.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:30:29.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:30:29.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:30:58.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:30:58.213 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:31:28.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:31:28.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:31:31.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:31:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:31:31.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:31:31.213 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:31:31.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:31:31.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2023-06-09T12:31:58.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:31:58.332 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:32:28.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:32:28.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:32:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:32:32.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:32:58.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:32:58.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:33:28.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:33:28.715 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:33:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:33:34.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:33:34.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:33:34.092 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:33:34.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:33:34.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2023-06-09T12:33:58.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:33:58.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:34:29.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:34:29.004 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:34:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:34:35.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:34:59.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:34:59.166 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:35:29.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:35:29.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:35:36.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:35:36.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:35:36.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:35:36.973 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:35:36.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:35:36.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2023-06-09T12:35:59.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:35:59.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:36:29.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:36:29.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:36:38.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:36:38.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:36:59.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:36:59.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:37:29.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:37:29.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:37:39.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:37:39.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:37:39.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:37:39.860 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:37:39.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:37:39.861 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2023-06-09T12:37:59.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:37:59.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:38:30.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:38:30.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:38:41.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:38:41.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:39:00.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:39:00.220 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:39:30.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:39:30.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:39:42.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:39:42.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:39:42.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:39:42.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:39:42.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:39:42.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2023-06-09T12:40:00.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:40:00.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:40:30.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:40:30.593 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:40:44.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:40:44.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:41:00.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:41:00.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:41:30.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:41:30.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:41:45.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:41:45.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:41:45.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:41:45.612 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:41:45.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:41:45.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2023-06-09T12:42:01.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:42:01.063 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:42:31.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:42:31.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:42:47.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:42:47.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:43:01.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:43:01.362 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:43:31.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:43:31.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:43:48.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:43:48.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:43:48.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:43:48.493 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:43:48.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:43:48.494 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2023-06-09T12:44:01.637 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:44:01.640 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:44:31.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:44:31.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:44:49.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:44:49.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:45:01.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:45:01.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:45:32.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:45:32.066 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:45:51.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:45:51.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:45:51.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:45:51.373 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:45:51.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:45:51.374 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2023-06-09T12:46:02.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:46:02.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:46:32.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:46:32.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:46:52.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:46:52.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:47:02.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:47:02.504 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:47:32.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:47:32.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:47:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:47:54.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:47:54.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:47:54.253 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:47:54.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:47:54.254 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2023-06-09T12:48:02.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:48:02.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:48:32.929 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:48:32.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:48:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:48:55.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:49:03.083 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:49:03.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:49:33.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:49:33.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:49:57.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:49:57.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:49:57.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:49:57.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:49:57.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:49:57.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2023-06-09T12:50:03.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:50:03.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:50:33.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:50:33.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:50:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:50:58.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:51:03.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:51:03.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:51:33.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:51:33.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:52:00.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:52:00.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:52:00.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:52:00.013 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:52:00.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:52:00.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2023-06-09T12:52:03.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:52:03.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:52:34.087 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:52:34.090 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:53:01.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:53:01.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:53:04.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:53:04.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:53:34.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:53:34.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:54:02.912 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:54:02.913 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:54:02.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:54:02.916 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:54:02.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:54:02.917 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2023-06-09T12:54:04.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:54:04.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:54:34.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:54:34.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:55:04.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:55:04.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:55:04.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:55:04.841 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:55:34.978 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:55:34.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:56:05.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:56:05.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:56:05.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:56:05.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:56:05.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:56:05.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:56:05.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:56:05.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2023-06-09T12:56:35.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:56:35.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:57:05.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:57:05.429 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:57:07.209 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:57:07.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:57:35.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:57:35.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:58:05.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:58:05.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:58:08.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:58:08.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T12:58:08.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:58:08.653 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T12:58:08.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T12:58:08.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2023-06-09T12:58:35.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:58:35.838 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:59:05.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:59:05.985 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:59:10.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T12:59:10.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T12:59:36.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:59:36.122 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:00:06.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:00:06.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:00:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:00:11.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:00:11.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:00:11.533 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:00:11.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:00:11.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2023-06-09T13:00:36.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:00:36.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:01:06.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:01:06.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:01:12.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:01:12.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:01:36.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:01:36.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:02:06.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:02:06.617 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:02:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:02:14.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:02:14.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:02:14.413 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:02:14.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:02:14.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2023-06-09T13:02:36.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:02:36.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:03:06.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:03:06.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:03:15.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:03:15.857 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:03:36.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:03:36.956 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:04:07.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:04:07.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:04:17.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:04:17.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:04:17.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:04:17.293 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:04:17.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:04:17.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2023-06-09T13:04:37.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:04:37.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:05:07.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:05:07.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:05:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:05:18.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:05:37.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:05:37.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:06:07.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:06:07.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:06:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:06:20.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:06:20.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:06:20.173 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:06:20.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:06:20.174 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2023-06-09T13:06:37.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:06:37.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:07:07.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:07:07.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:07:21.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:07:21.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:07:38.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:07:38.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:08:08.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:08:08.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:08:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:08:23.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:08:23.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:08:23.053 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:08:23.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:08:23.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2023-06-09T13:08:38.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:08:38.281 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:09:08.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:09:08.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:09:24.489 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:09:24.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:09:38.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:09:38.496 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:10:08.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:10:08.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:10:25.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:10:25.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:10:25.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:10:25.933 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:10:25.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:10:25.934 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2023-06-09T13:10:38.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:10:38.770 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:11:08.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:11:08.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:11:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:11:27.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:11:38.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:11:38.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:12:08.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:12:08.997 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:12:28.809 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:12:28.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:12:28.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:12:28.813 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:12:28.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:12:28.814 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2023-06-09T13:12:39.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:12:39.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:13:09.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:13:09.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:13:30.249 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:13:30.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:13:39.279 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:13:39.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:14:09.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:14:09.411 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:14:31.689 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:14:31.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:14:31.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:14:31.693 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:14:31.694 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:14:31.694 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2023-06-09T13:14:39.469 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:14:39.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:15:09.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:15:09.676 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:15:33.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:15:33.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:15:39.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:15:39.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:16:09.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:16:09.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:16:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:16:34.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:16:34.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:16:34.573 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:16:34.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:16:34.574 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2023-06-09T13:16:40.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:16:40.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:17:10.204 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:17:10.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:17:36.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:17:36.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:17:40.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:17:40.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:18:10.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:18:10.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:18:37.449 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:18:37.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:18:37.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:18:37.453 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:18:37.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:18:37.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2023-06-09T13:18:40.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:18:40.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:19:11.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:19:11.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:19:38.896 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:19:38.906 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:19:41.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:19:41.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:20:11.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:20:11.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:20:40.329 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:20:40.331 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:20:40.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:20:40.334 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:20:40.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:20:40.335 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2023-06-09T13:20:41.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:20:41.451 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:21:11.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:21:11.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:21:41.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:21:41.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:21:41.769 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:21:41.771 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:22:11.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:22:11.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:22:42.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:22:42.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:22:43.209 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:22:43.211 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:22:43.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:22:43.213 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:22:43.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:22:43.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2023-06-09T13:23:12.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:23:12.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:23:42.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:23:42.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:23:44.649 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:23:44.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:24:12.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:24:12.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:24:42.550 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:24:42.553 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:24:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:24:46.091 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:24:46.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:24:46.093 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:24:46.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:24:46.094 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2023-06-09T13:25:12.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:25:12.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:25:42.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:25:42.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:25:47.529 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:25:47.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:26:12.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:26:12.989 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:26:43.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:26:43.144 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:26:48.969 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:26:48.971 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:26:48.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:26:48.973 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:26:48.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:26:48.974 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2023-06-09T13:27:13.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:27:13.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:27:43.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:27:43.477 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:27:50.409 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:27:50.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:28:13.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:28:13.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:28:43.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:28:43.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:28:51.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:28:51.858 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:28:51.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:28:51.863 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:28:51.864 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:28:51.864 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2023-06-09T13:29:13.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:29:13.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:29:44.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:29:44.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:29:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:29:53.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:30:14.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:30:14.231 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:30:44.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:30:44.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:30:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:30:54.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:30:54.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:30:54.733 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:30:54.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:30:54.734 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2023-06-09T13:31:14.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:31:14.516 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:31:44.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:31:44.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:31:56.169 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:31:56.171 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:32:14.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:32:14.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:32:44.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:32:44.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:32:57.609 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:32:57.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:32:57.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:32:57.613 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:32:57.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:32:57.614 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2023-06-09T13:33:15.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:33:15.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:33:45.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:33:45.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:33:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:33:59.051 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:34:15.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:34:15.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:34:45.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:34:45.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:35:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:35:00.491 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:35:00.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:35:00.494 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:35:00.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:35:00.495 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2023-06-09T13:35:15.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:35:15.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:35:45.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:35:45.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:36:01.929 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:36:01.931 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:36:15.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:36:15.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:36:46.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:36:46.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:37:03.369 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:37:03.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:37:03.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:37:03.373 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:37:03.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:37:03.374 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2023-06-09T13:37:16.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:37:16.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:37:46.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:37:46.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:38:04.809 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:38:04.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:38:16.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:38:16.503 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:38:46.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:38:46.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:39:06.249 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:39:06.251 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:39:06.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:39:06.253 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:39:06.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:39:06.254 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2023-06-09T13:39:16.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:39:16.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:39:46.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:39:46.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:40:07.689 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:40:07.691 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:40:17.076 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:40:17.079 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:40:47.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:40:47.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:41:09.129 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:41:09.131 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:41:09.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:41:09.133 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:41:09.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:41:09.135 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2023-06-09T13:41:17.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:41:17.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:41:47.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:41:47.582 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:42:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:42:10.571 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:42:17.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:42:17.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:42:47.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:42:47.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:43:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:43:12.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:43:12.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:43:12.013 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:43:12.014 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:43:12.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2023-06-09T13:43:17.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:43:17.985 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:43:48.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:43:48.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:44:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:44:13.451 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:44:18.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:44:18.277 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:44:48.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:44:48.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:45:14.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:45:14.899 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:45:14.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:45:14.905 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:45:14.906 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:45:14.906 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2023-06-09T13:45:18.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:45:18.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:45:48.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:45:48.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:46:16.329 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:46:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:46:18.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:46:18.835 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:46:49.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:46:49.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:47:17.769 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:47:17.771 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:47:17.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:47:17.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:47:17.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:47:17.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2023-06-09T13:47:19.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:47:19.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:47:49.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:47:49.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:48:19.209 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:48:19.211 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:48:19.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:48:19.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:48:49.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:48:49.605 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:49:19.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:49:19.750 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:49:20.649 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:49:20.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:49:20.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:49:20.653 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:49:20.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:49:20.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2023-06-09T13:49:49.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:49:49.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:50:20.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:50:20.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:50:22.089 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:50:22.091 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:50:50.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:50:50.171 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:51:20.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:51:20.375 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:51:23.529 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:51:23.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:51:23.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:51:23.533 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:51:23.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:51:23.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2023-06-09T13:51:50.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:51:50.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:52:20.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:52:20.575 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:52:24.969 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:52:24.971 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:52:50.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:52:50.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:53:20.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:53:20.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:53:26.409 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:53:26.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:53:26.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:53:26.413 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:53:26.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:53:26.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2023-06-09T13:53:50.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:53:50.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:54:21.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:54:21.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:54:27.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:54:27.859 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:54:51.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:54:51.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:55:21.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:55:21.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:55:29.289 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:55:29.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:55:29.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:55:29.293 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:55:29.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:55:29.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2023-06-09T13:55:51.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:55:51.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:56:21.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:56:21.609 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:56:30.729 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:56:30.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:56:51.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:56:51.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:57:21.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:57:21.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:57:32.169 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:57:32.171 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:57:32.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:57:32.173 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:57:32.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:57:32.174 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2023-06-09T13:57:51.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:57:51.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:58:22.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:58:22.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:58:33.609 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:58:33.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T13:58:52.216 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:58:52.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:59:22.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:59:22.290 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:59:35.049 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T13:59:35.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T13:59:35.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T13:59:35.053 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T13:59:35.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T13:59:35.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2023-06-09T13:59:52.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T13:59:52.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:00:22.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:00:22.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:00:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:00:36.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:00:52.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:00:52.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:01:22.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:01:22.797 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:01:37.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:01:37.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:01:37.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:01:37.933 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:01:37.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:01:37.934 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2023-06-09T14:01:52.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:01:52.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:02:23.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:02:23.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:02:39.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:02:39.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:02:53.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:02:53.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:03:23.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:03:23.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:03:40.809 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:03:40.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:03:40.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:03:40.813 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:03:40.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:03:40.814 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2023-06-09T14:03:53.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:03:53.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:04:23.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:04:23.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:04:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:04:42.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:04:53.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:04:53.866 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:05:24.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:05:24.011 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:05:43.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:05:43.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:05:43.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:05:43.693 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:05:43.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:05:43.694 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2023-06-09T14:05:54.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:05:54.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:06:24.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:06:24.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:06:45.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:06:45.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:06:54.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:06:54.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:07:24.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:07:24.619 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:07:46.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:07:46.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:07:46.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:07:46.573 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:07:46.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:07:46.574 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2023-06-09T14:07:54.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:07:54.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:08:24.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:08:24.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:08:48.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:08:48.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:08:54.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:08:54.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:09:25.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:09:25.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:09:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:09:49.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:09:49.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:09:49.453 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:09:49.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:09:49.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2023-06-09T14:09:55.254 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:09:55.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:10:25.398 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:10:25.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:10:50.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:10:50.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:10:55.545 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:10:55.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:11:25.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:11:25.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:11:52.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:11:52.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:11:52.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:11:52.333 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:11:52.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:11:52.334 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2023-06-09T14:11:55.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:11:55.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:12:26.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:12:26.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:12:53.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:12:53.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:12:56.161 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:12:56.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:13:26.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:13:26.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:13:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:13:55.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:13:55.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:13:55.213 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:13:55.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:13:55.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2023-06-09T14:13:56.427 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:13:56.429 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:14:26.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:14:26.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:14:56.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:14:56.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:14:56.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:14:56.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:15:26.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:15:26.842 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:15:56.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:15:56.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:15:58.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:15:58.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:15:58.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:15:58.093 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:15:58.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:15:58.094 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2023-06-09T14:16:27.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:16:27.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:16:57.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:16:57.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:16:59.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:16:59.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:17:27.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:17:27.433 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:17:57.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:17:57.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:18:00.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:18:00.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:18:00.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:18:00.972 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:18:00.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:18:00.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2023-06-09T14:18:27.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:18:27.732 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:18:57.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:18:57.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:19:02.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:19:02.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:19:28.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:19:28.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:19:58.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:19:58.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:20:03.860 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:20:03.872 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:20:03.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:20:03.875 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:20:03.875 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:20:03.876 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2023-06-09T14:20:28.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:20:28.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:20:58.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:20:58.477 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:21:05.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:21:05.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:21:28.630 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:21:28.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:21:58.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:21:58.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:22:06.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:22:06.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:22:06.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:22:06.732 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:22:06.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:22:06.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2023-06-09T14:22:28.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:22:28.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:22:58.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:22:58.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:23:08.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:23:08.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:23:29.074 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:23:29.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:23:59.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:23:59.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:24:09.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:24:09.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:24:09.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:24:09.612 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:24:09.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:24:09.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2023-06-09T14:24:29.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:24:29.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:24:59.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:24:59.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:25:11.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:25:11.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:25:29.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:25:29.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:25:59.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:25:59.894 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:26:12.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:26:12.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:26:12.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:26:12.492 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:26:12.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:26:12.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2023-06-09T14:26:29.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:26:29.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:27:00.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:27:00.126 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:27:13.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:27:13.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:27:30.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:27:30.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:28:00.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:28:00.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:28:15.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:28:15.369 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:28:15.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:28:15.372 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:28:15.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:28:15.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2023-06-09T14:28:30.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:28:30.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:29:00.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:29:00.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:29:16.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:29:16.809 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:29:30.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:29:30.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:30:00.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:30:00.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:30:18.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:30:18.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:30:18.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:30:18.253 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:30:18.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:30:18.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2023-06-09T14:30:31.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:30:31.049 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:31:01.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:31:01.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:31:19.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:31:19.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:31:31.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:31:31.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:32:01.381 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:32:01.383 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:32:21.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:32:21.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:32:21.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:32:21.132 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:32:21.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:32:21.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2023-06-09T14:32:31.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:32:31.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:33:01.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:33:01.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:33:22.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:33:22.569 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:33:31.741 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:33:31.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:34:01.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:34:01.873 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:34:24.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:34:24.009 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:34:24.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:34:24.012 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:34:24.012 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:34:24.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2023-06-09T14:34:32.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:34:32.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:35:02.163 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:35:02.166 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:35:25.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:35:25.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:35:32.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:35:32.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:36:02.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:36:02.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:36:26.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:36:26.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:36:26.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:36:26.903 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:36:26.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:36:26.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2023-06-09T14:36:32.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:36:32.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:37:02.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:37:02.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:37:28.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:37:28.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:37:32.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:37:32.720 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:38:02.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:38:02.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:38:29.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:38:29.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:38:29.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:38:29.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:38:29.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:38:29.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2023-06-09T14:38:32.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:38:32.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:39:03.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:39:03.014 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:39:31.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:39:31.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:39:33.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:39:33.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:40:03.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:40:03.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:40:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:40:32.649 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:40:32.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:40:32.653 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:40:32.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:40:32.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2023-06-09T14:40:33.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:40:33.540 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:41:03.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:41:03.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:41:33.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:41:33.760 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:41:34.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:41:34.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:42:03.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:42:03.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:42:34.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:42:34.034 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:42:35.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:42:35.531 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:42:35.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:42:35.533 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:42:35.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:42:35.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2023-06-09T14:43:04.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:43:04.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:43:34.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:43:34.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:43:36.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:43:36.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:44:04.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:44:04.232 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:44:34.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:44:34.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:44:38.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:44:38.411 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:44:38.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:44:38.412 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:44:38.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:44:38.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2023-06-09T14:45:04.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:45:04.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:45:34.438 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:45:34.445 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:45:39.849 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:45:39.849 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:46:04.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:46:04.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:46:34.562 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:46:34.565 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:46:41.289 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:46:41.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:46:41.292 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:46:41.293 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:46:41.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:46:41.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2023-06-09T14:47:04.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:47:04.701 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:47:34.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:47:34.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:47:42.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:47:42.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:48:04.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:48:04.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:48:35.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:48:35.049 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:48:44.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:48:44.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:48:44.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:48:44.173 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:48:44.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:48:44.174 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2023-06-09T14:49:05.193 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:49:05.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:49:35.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:49:35.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:49:45.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:49:45.609 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:50:05.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:50:05.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:50:35.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:50:35.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:50:47.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:50:47.049 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:50:47.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:50:47.053 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:50:47.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:50:47.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2023-06-09T14:51:05.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:51:05.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:51:35.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:51:35.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:51:48.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:51:48.489 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:52:06.024 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:52:06.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:52:36.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:52:36.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:52:49.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:52:49.929 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:52:49.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:52:49.932 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:52:49.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:52:49.933 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2023-06-09T14:53:06.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:53:06.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:53:36.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:53:36.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:53:51.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:53:51.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:54:06.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:54:06.619 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:54:36.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:54:36.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:54:52.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:54:52.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:54:52.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:54:52.813 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:54:52.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:54:52.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2023-06-09T14:55:06.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:55:06.893 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:55:37.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:55:37.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:55:54.248 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:55:54.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:56:07.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:56:07.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:56:37.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:56:37.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:56:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:56:55.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:56:55.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:56:55.693 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:56:55.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:56:55.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2023-06-09T14:57:07.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:57:07.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:57:37.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:57:37.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:57:57.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:57:57.129 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T14:58:07.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:58:07.768 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:58:37.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:58:37.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:58:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T14:58:58.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T14:58:58.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T14:58:58.572 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T14:58:58.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T14:58:58.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2023-06-09T14:59:08.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:59:08.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:59:38.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T14:59:38.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:00:00.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:00:00.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:00:08.343 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:00:08.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:00:38.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:00:38.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:01:01.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:01:01.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:01:01.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:01:01.453 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:01:01.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:01:01.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2023-06-09T15:01:08.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:01:08.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:01:38.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:01:38.963 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:02:02.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:02:02.897 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:02:09.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:02:09.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:02:39.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:02:39.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:03:04.328 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:03:04.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:03:04.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:03:04.332 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:03:04.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:03:04.333 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2023-06-09T15:03:09.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:03:09.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:03:39.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:03:39.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:04:05.768 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:04:05.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:04:09.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:04:09.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:04:40.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:04:40.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:05:07.208 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:05:07.210 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:05:07.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:05:07.213 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:05:07.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:05:07.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2023-06-09T15:05:10.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:05:10.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:05:40.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:05:40.411 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:06:08.648 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:06:08.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:06:10.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:06:10.541 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:06:40.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:06:40.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:07:10.088 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:07:10.090 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:07:10.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:07:10.092 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:07:10.093 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:07:10.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2023-06-09T15:07:10.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:07:10.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:07:40.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:07:40.969 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:08:11.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:08:11.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:08:11.528 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:08:11.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:08:41.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:08:41.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:09:11.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:09:11.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:09:12.968 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:09:12.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:09:12.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:09:12.974 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:09:12.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:09:12.975 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2023-06-09T15:09:41.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:09:41.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:10:11.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:10:11.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:10:14.408 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:10:14.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:10:41.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:10:41.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:11:12.052 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:11:12.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:11:15.848 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:11:15.850 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:11:15.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:11:15.853 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:11:15.853 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:11:15.854 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2023-06-09T15:11:42.190 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:11:42.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:12:12.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:12:12.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:12:17.288 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:12:17.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:12:42.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:12:42.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:13:12.651 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:13:12.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:13:18.728 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:13:18.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:13:18.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:13:18.733 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:13:18.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:13:18.734 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2023-06-09T15:13:42.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:13:42.784 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:14:12.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:14:12.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:14:20.168 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:14:20.170 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:14:43.090 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:14:43.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:15:13.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:15:13.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:15:21.608 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:15:21.610 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:15:21.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:15:21.613 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:15:21.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:15:21.614 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2023-06-09T15:15:43.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:15:43.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:16:13.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:16:13.496 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:16:23.048 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:16:23.050 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:16:43.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:16:43.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:17:13.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:17:13.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:17:24.488 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:17:24.490 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:17:24.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:17:24.493 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:17:24.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:17:24.494 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2023-06-09T15:17:43.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:17:43.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:18:14.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:18:14.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:18:25.928 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:18:25.930 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:18:44.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:18:44.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:19:14.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:19:14.296 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:19:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:19:27.370 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:19:27.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:19:27.373 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:19:27.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:19:27.374 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2023-06-09T15:19:44.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:19:44.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:20:14.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:20:14.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:20:28.808 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:20:28.810 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:20:44.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:20:44.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:21:14.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:21:14.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:21:30.249 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:21:30.250 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:21:30.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:21:30.253 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:21:30.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:21:30.254 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2023-06-09T15:21:44.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:21:44.794 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:22:14.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:22:14.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:22:31.688 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:22:31.690 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:22:44.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:22:44.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:23:14.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:23:14.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:23:33.128 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:23:33.130 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:23:33.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:23:33.133 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:23:33.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:23:33.134 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2023-06-09T15:23:45.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:23:45.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:24:15.135 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:24:15.138 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:24:34.568 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:24:34.570 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:24:45.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:24:45.208 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:25:15.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:25:15.352 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:25:36.008 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:25:36.010 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:25:36.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:25:36.013 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:25:36.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:25:36.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2023-06-09T15:25:45.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:25:45.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:26:15.615 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:26:15.618 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:26:37.448 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:26:37.450 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:26:45.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:26:45.772 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:27:15.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:27:15.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:27:38.895 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:27:38.898 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:27:38.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:27:38.904 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:27:38.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:27:38.905 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2023-06-09T15:27:46.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:27:46.061 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:28:16.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:28:16.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:28:40.329 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:28:40.330 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:28:46.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:28:46.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:29:16.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:29:16.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:29:41.769 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:29:41.770 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:29:41.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:29:41.773 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:29:41.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:29:41.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2023-06-09T15:29:46.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:29:46.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:30:16.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:30:16.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:30:43.209 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:30:43.211 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:30:46.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:30:46.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:31:16.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:31:17.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:31:44.649 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:31:44.650 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:31:44.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:31:44.653 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:31:44.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:31:44.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2023-06-09T15:31:47.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:31:47.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:32:17.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:32:17.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:32:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:32:46.091 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:32:47.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:32:47.423 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:33:17.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:33:17.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:33:47.529 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:33:47.530 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:33:47.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:33:47.533 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:33:47.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:33:47.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2023-06-09T15:33:47.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:33:47.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:34:17.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:34:17.841 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:34:47.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:34:47.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:34:48.969 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:34:48.970 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:35:18.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:35:18.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:35:48.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:35:48.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:35:50.409 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:35:50.410 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:35:50.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:35:50.413 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:35:50.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:35:50.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2023-06-09T15:36:18.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:36:18.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:36:48.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:36:48.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:36:51.855 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:36:51.858 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:37:18.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:37:18.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:37:48.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:37:48.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:37:53.289 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:37:53.290 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:37:53.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:37:53.293 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:37:53.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:37:53.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2023-06-09T15:38:18.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:38:18.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:38:48.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:38:48.878 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:38:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:38:54.730 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:39:18.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:39:18.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:39:49.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:39:49.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:39:56.169 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:39:56.171 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:39:56.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:39:56.174 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:39:56.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:39:56.175 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2023-06-09T15:40:19.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:40:19.232 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:40:49.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:40:49.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:40:57.609 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:40:57.611 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:41:19.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:41:19.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:41:49.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:41:49.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:41:59.049 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:41:59.051 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:41:59.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:41:59.054 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:41:59.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:41:59.055 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2023-06-09T15:42:19.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:42:19.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:42:49.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:42:49.650 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:43:00.489 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:43:00.491 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:43:19.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:43:19.720 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:43:49.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:43:49.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:44:01.929 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:44:01.931 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:44:01.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:44:01.934 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:44:01.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:44:01.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2023-06-09T15:44:19.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:44:19.856 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:44:50.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:44:50.014 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:45:03.369 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:45:03.371 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:45:20.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:45:20.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:45:50.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:45:50.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:46:04.809 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:46:04.811 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:46:04.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:46:04.813 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:46:04.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:46:04.814 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2023-06-09T15:46:20.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:46:20.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:46:50.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:46:50.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:47:06.249 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:47:06.251 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:47:20.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:47:20.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:47:50.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:47:50.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:48:07.689 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:48:07.691 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:48:07.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:48:07.694 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:48:07.694 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:48:07.695 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2023-06-09T15:48:21.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:48:21.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:48:51.081 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:48:51.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:49:09.129 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:49:09.131 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:49:21.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:49:21.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:49:51.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:49:51.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:50:10.569 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:50:10.571 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:50:10.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:50:10.574 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:50:10.574 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:50:10.575 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2023-06-09T15:50:21.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:50:21.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:50:51.645 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:50:51.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:51:12.009 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:51:12.011 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:51:21.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:51:21.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:51:51.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:51:51.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:52:13.449 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:52:13.451 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:52:13.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:52:13.454 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:52:13.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:52:13.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2023-06-09T15:52:21.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:52:21.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:52:52.117 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:52:52.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:53:14.896 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:53:14.898 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:53:22.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:53:22.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:53:52.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:53:52.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:54:16.329 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:54:16.331 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:54:16.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:54:16.334 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:54:16.335 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:54:16.335 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2023-06-09T15:54:22.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:54:22.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:54:52.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:54:52.526 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:55:17.769 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:55:17.771 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:55:22.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:55:22.637 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:55:52.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:55:52.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:56:19.210 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:56:19.212 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:56:19.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:56:19.214 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:56:19.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:56:19.215 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2023-06-09T15:56:22.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:56:22.861 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:56:52.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:56:52.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:57:20.650 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:57:20.651 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:57:23.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:57:23.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:57:53.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:57:53.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:58:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:58:22.091 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T15:58:22.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T15:58:22.094 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T15:58:22.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T15:58:22.095 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2023-06-09T15:58:23.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:58:23.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:58:53.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:58:53.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:59:23.530 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T15:59:23.531 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T15:59:23.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:59:23.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:59:53.774 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T15:59:53.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:00:23.928 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:00:23.931 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:00:24.970 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:00:24.971 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:00:24.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:00:24.974 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:00:24.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:00:24.975 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2023-06-09T16:00:54.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:00:54.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:01:24.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:01:24.270 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:01:26.410 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:01:26.411 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:01:54.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:01:54.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:02:24.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:02:24.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:02:27.856 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:02:27.859 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:02:27.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:02:27.862 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:02:27.863 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:02:27.863 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2023-06-09T16:02:54.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:02:54.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:03:24.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:03:24.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:03:29.290 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:03:29.291 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:03:54.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:03:54.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:04:24.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:04:24.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:04:30.730 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:04:30.732 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:04:30.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:04:30.735 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:04:30.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:04:30.736 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2023-06-09T16:04:55.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:04:55.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:05:25.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:05:25.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:05:32.170 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:05:32.172 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:05:55.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:05:55.152 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:06:25.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:06:25.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:06:33.610 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:06:33.612 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:06:33.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:06:33.615 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:06:33.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:06:33.616 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2023-06-09T16:06:55.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:06:55.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:07:25.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:07:25.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:07:35.050 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:07:35.052 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:07:55.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:07:55.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:08:25.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:08:25.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:08:36.490 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:08:36.492 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:08:36.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:08:36.495 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:08:36.495 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:08:36.496 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2023-06-09T16:08:55.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:08:55.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:09:25.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:09:25.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:09:37.930 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:09:37.931 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:09:56.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:09:56.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:10:26.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:10:26.157 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:10:39.370 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:10:39.372 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:10:39.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:10:39.374 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:10:39.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:10:39.375 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2023-06-09T16:10:56.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:10:56.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:11:26.437 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:11:26.440 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:11:40.810 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:11:40.812 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:11:56.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:11:56.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:12:26.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:12:26.715 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:12:42.250 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:12:42.252 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:12:42.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:12:42.255 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:12:42.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:12:42.256 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2023-06-09T16:12:56.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:12:56.869 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:13:27.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:13:27.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:13:43.690 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:13:43.692 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:13:57.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:13:57.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:14:27.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:14:27.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:14:45.130 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:14:45.132 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:14:45.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:14:45.134 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:14:45.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:14:45.135 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2023-06-09T16:14:57.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:14:57.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:15:27.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:15:27.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:15:46.570 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:15:46.572 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:15:57.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:15:57.867 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:16:28.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:16:28.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:16:48.010 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:16:48.012 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:16:48.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:16:48.015 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:16:48.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:16:48.015 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2023-06-09T16:16:58.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:16:58.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:17:28.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:17:28.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:17:49.450 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:17:49.452 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:17:58.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:17:58.452 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:18:28.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:18:28.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:18:50.897 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:18:50.899 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:18:50.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:18:50.903 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:18:50.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:18:50.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2023-06-09T16:18:58.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:18:58.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:19:28.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:19:28.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:19:52.330 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:19:52.332 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:19:59.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:19:59.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:20:29.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:20:29.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:20:53.770 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:20:53.772 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:20:53.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:20:53.775 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:20:53.775 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:20:53.776 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2023-06-09T16:20:59.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:20:59.255 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:21:29.317 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:21:29.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:21:55.211 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:21:55.212 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:21:59.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:21:59.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:22:29.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:22:29.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:22:56.650 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:22:56.652 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: could not mount 172.21.15.92:6789:/test on /tmp/tmp.dvoqC6rw3jtest 2023-06-09T16:22:56.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T16:22:56.654 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-09T16:22:56.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-09T16:22:56.655 DEBUG:teuthology.orchestra.run.smithi092:> (cd /tmp/tmp.3HuE1HNmdOxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2023-06-09T16:22:59.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:22:59.640 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:23:29.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:23:29.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:23:58.090 INFO:teuthology.orchestra.run.smithi092.stderr:mount error 110 = Connection timed out 2023-06-09T16:23:58.092 INFO:teuthology.orchestra.run.smithi092.stdout:common/rc: retrying test device mount with external set 2023-06-09T16:23:59.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:23:59.894 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T16:24:24.851 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-06-09T16:24:24.851 DEBUG:teuthology.task.console_log:Killing console logger for smithi092 2023-06-09T16:24:24.852 DEBUG:teuthology.task.console_log:Killing console logger for smithi123 2023-06-09T16:24:24.852 DEBUG:teuthology.exit:Finished running handlers